Image data processing method, computer device, and storage medium

Abstract

The present application relates to a method for processing image data, a computer device, and a storage medium. The method includes: the computer device inputs medical image data including organs, blood vessels, and lesions into different segmentation models respectively, obtains the segmentation results corresponding to each segmentation model, performs three-dimensional reconstruction according to each segmentation result to obtain a reconstructed image including organs, blood vessels, and lesions, constructs a surface model of the lesion based on the reconstructed image, and performs a simulated resection of the lesion based on the surface model. In this solution, the segmentation results corresponding to each segmentation model are obtained based on the segmentation models corresponding to the organs, blood vessels, and lesions respectively, and the segmentation results are relatively accurate. The reconstructed images obtained based on each segmentation result are also relatively precise; by constructing a surface model of the lesion area based on the reconstructed image, the lesion can be visually observed based on the surface model, and a simulated resection of the lesion can be performed based on the surface model, providing effective data support for the user's lesion resection operation.

Description

Technical Field

[0001] This application relates to the field of medical technology, and in particular, to an image data processing method, a computer device, and a storage medium. Background Art

[0002] Organ tumor resection is a common medical technique. For example, liver tumor resection, lung tumor (lung cancer) resection, etc. Generally, tumor resection involves two aspects of technical difficulties. On the one hand, three-dimensional reconstruction needs to be performed based on the spatial position relationship of blood vessels, organs, and lesions. On the other hand, actual tumor resection surgery has relatively high risks, and the tumor resection plan plays a crucial role.

[0003] Exemplarily, in the scenario of liver tumor surgical resection, for the three-dimensional reconstruction of the spatial position relationship of blood vessels, liver segments, and lesions, the traditional three-dimensional reconstruction method of blood vessels, liver segments, and lesions is to manually and semi-automatically outline the blood vessels and liver segment parts in the liver image frame by frame by the user. For example, it is necessary to outline the liver segments according to the blood vessel directions of the hepatic veins and portal veins.

[0004] The above method of manually outlining tumors to extract image features is inefficient and has limitations, and cannot provide reliable data support for tumor resection surgery. Summary of the Invention

[0005] Based on this, it is necessary to provide an image data processing method, a computer device, and a storage medium that can provide data support for resection surgery for the above technical problems.

[0006] In a first aspect, this application provides an image data processing method. The method includes:

[0007] Input medical image data into different segmentation models respectively to obtain segmentation results corresponding to each segmentation model; the medical image data includes organs, blood vessels, and lesions;

[0008] Perform three-dimensional reconstruction based on each segmentation result to obtain a reconstructed image; the reconstructed image includes organs, blood vessels, and lesions;

[0009] Construct a surface model of the lesion based on the reconstructed image, and perform a simulated resection of the lesion based on the surface model.

[0010] In one optional embodiment, constructing a surface model of the lesion based on the reconstructed image includes:

[0011] Based on the position of the lesion in the reconstructed image, determine the area to be processed corresponding to the lesion on the organ;

[0012] Set at least three initial control points in the area to be processed, and generate a surface model based on the at least three initial control points.

[0013] In one alternative embodiment, at least three initial control points are set in the area to be processed, and a surface model is generated based on the at least three initial control points, including:

[0014] Generate an initial plane model according to the at least three initial control points;

[0015] Perform rotation and translation operations on the initial plane model according to a preset rotation direction, rotation displacement, and the resolution of the initial plane model, and obtain target control points corresponding to the initial plane model during the rotation and translation operations;

[0016] Generate a surface model according to the target control points and a surface reconstruction algorithm.

[0017] In one alternative embodiment, the method further includes:

[0018] Obtain the distances between the voxel points of the surface model and the corresponding voxel points in the area to be excised;

[0019] If the distances are outside a preset distance threshold range, adjust the surface model to obtain an adjusted surface model;

[0020] Simulate excising the lesion based on the surface model, including:

[0021] Simulate excising the lesion based on the adjusted surface model.

[0022] In one alternative embodiment, adjusting the surface model to obtain an adjusted surface model includes:

[0023] Adjust the positions of the target control points so that the distances between the target control points and the corresponding voxel points in the area to be excised are within the distance threshold range;

[0024] Generate an adjusted surface model according to the adjusted target control points and a surface reconstruction algorithm.

[0025] In one alternative embodiment, adjusting the surface model to obtain an adjusted surface model includes:

[0026] Determine a target area within a preset radius range in the surface model based on the voxel points; the target area contains the voxel points;

[0027] Obtain a plurality of candidate control points of the target area according to the resolution of the target area;

[0028] Adjust the positions of the candidate control points so that the distances between the candidate control points and the corresponding voxel points in the area to be processed are within the distance threshold range;

[0029] Based on multiple adjusted candidate control points and a surface reconstruction algorithm, an adjusted surface model is obtained.

[0030] In one optional embodiment, the medical image data includes venous phase image data and arterial phase image data; the medical image data is respectively input into different segmentation models to obtain the segmentation results corresponding to each segmentation model, including:

[0031] The arterial phase image data is input into the first blood vessel segmentation model to obtain an arterial blood vessel segmentation result;

[0032] The venous phase image data is input into the second blood vessel segmentation model to obtain a venous blood vessel segmentation result;

[0033] The venous phase image data and the venous blood vessel segmentation result are input into the organ segmentation model to obtain an organ segmentation result;

[0034] The arterial phase image data and the venous phase image data are input into the lesion segmentation model to obtain a lesion segmentation result.

[0035] In one optional embodiment, the organ segmentation model includes a first organ segmentation sub-model and a second organ segmentation sub-model; the venous phase image data and the venous blood vessel segmentation result are input into the organ segmentation model to obtain an organ segmentation result, including:

[0036] The venous phase image data is input into the first organ segmentation sub-model to obtain a segmentation result of the first organ;

[0037] The venous blood vessel segmentation result, the segmentation result of the first organ, and the venous phase image data are input into the second organ segmentation sub-model to obtain a segmentation result of the second organ; the second organ is included in the first organ.

[0038] In one optional embodiment, 3D reconstruction is performed based on each segmentation result to obtain a reconstructed image, including:

[0039] Based on the venous phase image data, the arterial phase image data, and an image registration algorithm, the arterial blood vessel segmentation result is registered to obtain a registered arterial blood vessel segmentation result;

[0040] 3D reconstruction is performed according to the registered arterial blood vessel segmentation result, the venous blood vessel segmentation result, the lesion segmentation result, and the segmentation result of the second organ to obtain a reconstructed image.

[0041] In one optional embodiment, based on the venous phase image data, the arterial phase image data, and an image registration algorithm, the arterial blood vessel segmentation result is registered to obtain a registered arterial blood vessel segmentation result, including:

[0042] Using the venous phase image data as the reference image and the arterial phase image data as the floating image, determine the registration transformation matrix between the venous phase image data and the arterial phase image data;

[0043] Map the arterial blood vessel segmentation result into the registration transformation matrix to obtain the segmented result of the arterial blood vessels after registration.

[0044] In one alternative embodiment, the method further includes:

[0045] Based on the surface model, simulate the resection of the lesion to obtain the resected area and the remaining area;

[0046] According to the resected area and the remaining area, determine the quantitative information of the organ; the quantitative information is used to represent the volume ratio of the remaining area to the overall area of the organ.

[0047] In a second aspect, the present application also provides an image data processing device. The device includes:

[0048] A segmentation module, configured to input medical image data into different segmentation models respectively to obtain the segmentation results corresponding to the respective segmentation models; the medical image data includes organs, blood vessels, and lesions;

[0049] A three-dimensional reconstruction module, configured to perform three-dimensional reconstruction according to the respective segmentation results to obtain a reconstructed image; the reconstructed image includes organs, blood vessels, and lesions;

[0050] A simulation module, configured to construct a surface model of the lesion based on the reconstructed image and simulate the resection of the lesion based on the surface model.

[0051] In a third aspect, the present application also provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the image data processing method provided in the first aspect.

[0052] In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the image data processing method provided in the first aspect.

[0053] In a fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the image data processing method provided in the first aspect.

[0054] The above-mentioned medical image data processing method, device, computer equipment and storage medium. The computer equipment inputs medical image data including organs, blood vessels and lesions into different segmentation models respectively, obtains the segmentation results corresponding to each segmentation model, performs three-dimensional reconstruction based on each segmentation result to obtain a reconstructed image including organs, blood vessels and lesions, constructs a curved surface model of the lesion based on the reconstructed image, and performs simulated resection of the lesion based on the curved surface model. In this solution, the computer equipment first obtains the segmentation results corresponding to each segmentation model based on the segmentation models corresponding to the organs, blood vessels and lesions respectively, performs three-dimensional reconstruction processing based on each segmentation result to obtain a reconstructed image, and the reconstructed image includes the spatial position relationship between the organs, blood vessels and lesions. There is no need for the user to manually outline the features of the organs, blood vessels and lesions, which reduces the time for three-dimensional reconstruction of the image. In addition, by constructing a curved surface model of the lesion area based on the reconstructed image, the lesion can be visually observed based on the curved surface model, and the position, direction, size, etc. of the curved surface model can be determined. Therefore, the simulated resection of the lesion based on the curved surface model is more accurate. The simulated resection plan obtained based on the simulated resection can provide effective data support for the user's lesion resection operation, further improving the efficiency and accuracy of lesion resection. Description of the Drawings

[0055] Figure 1 It is an application environment diagram of the medical image data processing method in an embodiment;

[0056] Figure 2 It is a schematic flowchart of the medical image data processing method in an embodiment;

[0057] Figure 3 It is a schematic flowchart of the medical image data processing method in another embodiment;

[0058] Figure 4 It is a schematic flowchart of the medical image data processing method in another embodiment;

[0059] Figure 5 It is a schematic plan view of the plane formed by control points in an embodiment;

[0060] Figure 6 It is a schematic flowchart of the medical image data processing method in another embodiment;

[0061] Figure 7 It is a schematic diagram of the preliminary cutting plane in the medical image data processing method in an embodiment;

[0062] Figure 8 It is a schematic diagram of adjusting the positioning and cutting curved surface model in the medical image data processing method in an embodiment;

[0063] Figure 9 It is a schematic diagram of curved surface cutting in the medical image data processing method in an embodiment;

[0064] Figure 10 It is a schematic flowchart of an image data processing method in another embodiment;

[0065] Figure 11 It is a schematic flowchart of an image data processing method in another embodiment;

[0066] Figure 12 It is a schematic flowchart of the segmentation process in an image data processing method in one embodiment;

[0067] Figure 13 It is a schematic flowchart of an image data processing method in another embodiment;

[0068] Figure 14 It is a schematic diagram of the three-dimensional reconstruction results of organs, blood vessels and lesions in one embodiment;

[0069] Figure 15 It is a schematic flowchart of an image data processing method in another embodiment;

[0070] Figure 16 It is a schematic flowchart of an image data processing method in another embodiment;

[0071] Figure 17 It is a schematic flowchart of an image data processing method in another embodiment;

[0072] Figure 18 It is a structural block diagram of an image data processing device in one embodiment;

[0073] Figure 19 It is a structural block diagram of an image data processing device in another embodiment. Detailed implementation manners

[0074] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0075] The image data processing method provided in this embodiment can be applied to the scene of simulated resection of lesions corresponding to any organ or any part. In this embodiment, the lesion is taken as a liver tumor as an example for illustration.

[0076] Malignant liver tumors are liver diseases with high incidence and fatality rates in recent years. Surgical resection of liver tumors is the main means of treating liver tumors. In the technology for liver tumor resection, on the one hand, it is necessary to accurately reconstruct the relationship between blood vessels, liver segments and the spatial position of liver tumors; on the other hand, it is necessary to accurately grasp the resection space of the liver tumor from the blood vessels and determine the simulated path for liver tumor resection.

[0077] Traditional three-dimensional reconstruction of blood vessels, liver segments, and liver tumors involves manually and semi-automatically outlining liver imaging frames one by one by users. Especially for outlining the characteristics of blood vessels and liver segments, since the outlining of liver segments is based on the blood vessel directions of hepatic veins and portal veins, the manual communication efficiency within liver segments is relatively low. Semi-automated methods, including methods based on threshold segmentation, region growing, atlases, etc., can improve the efficiency of three-dimensional reconstruction. However, these methods all require manual extraction of image features and have limitations.

[0078] Therefore, this embodiment proposes an image data processing method. Based on deep learning network technology, it realizes the precise segmentation of the liver, blood vessels, and lesions, and thus constructs three-dimensional reconstruction images of the liver, blood vessels, and lesions based on the segmentation results. A curved surface model of the lesion area is constructed based on the three-dimensional reconstruction images of the liver, blood vessels, and lesions. Based on the curved surface model, the position of the lesion can be intuitively observed, thereby realizing the precise simulated resection of the lesion. The image data processing method provided in this embodiment can provide sufficient basis for surgical planning for doctors, and greatly save doctors' surgical time and improve the efficiency and accuracy of lesion resection through automatic precise three-dimensional reconstruction and precise surgical planning.

[0079] The image data processing method provided in the embodiments of this application can be applied to an application environment as Figure 1 shown. In one embodiment, a computer device is provided. This computer device can be a terminal, and its internal structure diagram can be as Figure 1 shown. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it realizes an image data processing method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, trackball, or touchpad set on the outer shell of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0080] Those skilled in the art can understand that Figure 1The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0081] In one embodiment, as Figure 2 shown, a method for processing medical image data is provided. Taking the computer device in Figure 1 as an example for illustration, the method includes the following steps:

[0082] Step 201: Input the medical image data into different segmentation models respectively to obtain the segmentation results corresponding to each segmentation model; the medical image data includes organs, blood vessels, and lesions.

[0083] Among them, the medical image data can be enhanced Computed Tomography (CT) sequence images, Magnetic Resonance (MR) images, or medical images of other modalities. Exemplarily, taking the organ as the liver and the lesion as a liver tumor as an example, the computer device can obtain abdominal enhanced CT sequence images, divide them into different phase data according to the attribute values of each image in the abdominal enhanced CT sequence images, and determine the venous phase image data and arterial phase image data in the medical image data based on the image features and phase attributes. Correspondingly, the segmentation models can include a venous blood vessel segmentation model, an arterial blood vessel segmentation model, a liver segmentation model, a liver segment segmentation model, a liver tumor segmentation model, and so on. Of course, if the organ is the lung and the lesion can be a pulmonary nodule, and the medical image data can be chest enhanced CT sequence images, then correspondingly, the segmentation models can include a venous blood vessel segmentation model, an arterial blood vessel segmentation model, a lung lobe segment segmentation model, a trachea segmentation model, a pulmonary nodule segmentation model, and so on.

[0084] In this embodiment, the computer device inputs the medical image data corresponding to the organ into different segmentation models to obtain the segmentation results corresponding to different segmentation models. Exemplarily, taking the organ as the liver and the lesion as a liver tumor as an example, when the medical image data is input into the venous blood vessel segmentation model, the venous blood vessel segmentation result can be obtained; when the medical image data is input into the arterial blood vessel segmentation model, the arterial blood vessel segmentation result can be obtained; when the medical image data is input into the liver segmentation model, the liver segmentation result can be obtained; when the medical image data is input into the liver segment segmentation model, the liver segment segmentation result can be obtained; when the medical image data is input into the liver tumor segmentation model, the liver tumor segmentation result can be obtained, etc. This embodiment does not limit this.

[0085] Step 202: Perform three-dimensional reconstruction based on each segmentation result to obtain a reconstructed image; the reconstructed image includes organs, blood vessels, and lesions.

[0086] In this embodiment, after the computer device obtains the segmentation results output based on different segmentation models, it performs three-dimensional reconstruction processing based on each segmentation result, so as to obtain a reconstructed image including each segmentation result. Among them, the computer device can use VTK medical three-dimensional reconstruction technology to perform three-dimensional reconstruction on the segmentation results. Optionally, a slice two-dimensional visualization image can also be obtained based on the reconstructed image, which is not limited in this embodiment.

[0087] Taking the above-mentioned organ as the liver and the lesion as a liver tumor as an example, the computer device performs three-dimensional reconstruction processing on the venous blood vessel segmentation result, arterial blood vessel segmentation result, liver segmentation result, liver segment segmentation result, and liver tumor segmentation result obtained based on different segmentation models to obtain a three-dimensional visual reconstruction image including the relationship between the venous blood vessels, arterial blood vessels, liver, liver segments, and liver tumors, which is not limited in this embodiment.

[0088] Step 203: Construct a surface model of the lesion based on the reconstructed image, and perform simulated resection on the lesion based on the surface model.

[0089] In this embodiment, the reconstructed image contains the spatial position relationship between the organs, blood vessels, and the lesion. After the computer device obtains the reconstructed image, it can determine the position information of the lesion based on the reconstructed image, so as to construct a surface model corresponding to the area where the lesion is located. Among them, the construction algorithm of the surface model can determine the curve of the area where the lesion is located based on the voxel points in the area where the lesion is located in the reconstructed image, so as to perform modeling of the surface model based on the curve, which is not limited in this embodiment.

[0090] After performing surface modeling on the area where the lesion is located, the computer device can input the reconstructed image containing the surface model into the resection simulation software to perform simulated resection processing on the area to be processed corresponding to the surface model, and obtain information such as the resection direction, resection angle, and resection path during the simulated resection process. After completing the simulated resection, information such as the resection volume and remaining volume corresponding to the organ can also be obtained, so as to determine the simulated resection plan for the lesion of the organ. Optionally, the computer device uses the adjusted surface simulation scalpel to perform resection based on the surface model corresponding to the lesion through the resection simulation software to more adaptively fit the spatial shape of the surface model, which is not limited in this embodiment.

[0091] In the above image data processing method, the computer device inputs medical image data including organs, blood vessels, and lesions into different segmentation models respectively, obtains the segmentation results corresponding to each segmentation model, performs three-dimensional reconstruction processing based on each segmentation result to obtain a reconstructed image including organs, blood vessels, and lesions, constructs a surface model of the lesion based on the reconstructed image, and performs a simulated resection of the lesion based on the surface model. In this solution, the computer device first obtains the segmentation results corresponding to each segmentation model based on the segmentation models corresponding to the organs, blood vessels, and lesions respectively, and the reconstructed image obtained by performing three-dimensional reconstruction processing based on each segmentation result. The spatial position relationship between the organs, blood vessels, and lesions is included in the reconstructed image, and there is no need for the user to manually outline the features of the organs, blood vessels, and lesions, which reduces the time for the reconstructed image. In addition, by constructing a surface model of the lesion area based on the reconstructed image, the lesion can be visually observed based on the surface model, and the position, orientation, size, etc. of the surface model can be determined, so that the simulated resection of the lesion based on the surface model is more accurate. The simulated resection plan obtained based on the simulated resection can provide effective data support for the user's lesion resection operation, further improving the efficiency and accuracy of the lesion resection.

[0092] After the computer device obtains the reconstructed image, a surface model corresponding to the lesion area in the reconstructed image can be determined based on a preset surface construction algorithm. In one optional embodiment, as Figure 3 shown, constructing a surface model of the lesion based on the reconstructed image includes:

[0093] Step 301, based on the position of the lesion in the reconstructed image, determine the area to be processed corresponding to the lesion on the organ.

[0094] In this embodiment, after the computer device obtains the reconstructed image including the organs, blood vessels, and lesions, based on the spatial position of the lesion in the reconstructed image, on the organ in the reconstructed image, determine the area to be processed corresponding to the lesion, that is, determine the area to be processed in the reconstructed image. Taking the organ as the liver and the lesion as a liver tumor as an example, the computer device determines the position of the liver tumor on the liver in the three-dimensional visualization reconstructed image through the three-dimensional visualization reconstructed image of the liver, blood vessels, and liver tumor, and further determines the position of the liver tumor on the liver segment in the three-dimensional visualization reconstructed image based on the position of the liver tumor on the liver, so as to determine the liver segment where the liver tumor is located in the three-dimensional visualization reconstructed image as the area to be processed.

[0095] Step 302, set at least three initial control points in the area to be processed, and generate a surface model based on the at least three initial control points.

[0096] In this embodiment, taking the organ as the liver and the lesion as a liver tumor as an example, after the computer device obtains the area to be processed including the liver segment where the liver tumor is located, at least three initial control points are determined based on the area to be processed. Here, the purpose of determining the control points is to generate an initial plane. Therefore, the number of initial control points can be three. Optionally, the computer device can determine the initial control points at the spatial position corresponding to the area to be processed, or can obtain the initial control points determined by the user through interaction with the user. Exemplarily, the computer device can construct an initial plane through three initial control points, and then perform operations such as rotation and translation around the area to be processed according to the initial plane to form a surface model corresponding to the area to be processed. In addition, the computer device can also input the positions of the initial control points into a preset surface construction model to obtain the surface model corresponding to the area to be processed. This embodiment does not make any limitations in this regard.

[0097] In this embodiment, the computer device can construct a surface model including the lesion area based on the reconstructed images of the organ, blood vessels and lesions. Thus, the computer device can perform simulated resection of the lesion based on the surface model, providing effective data support for the lesion resection plan. Moreover, the surface model can very intuitively reflect the position and spatial state of the lesion, improving the accuracy of lesion resection.

[0098] One specific implementation method for the computer device to generate a surface model based on the initial control points is that in one alternative embodiment, as Figure 4 shown, at least three initial control points are set in the area to be processed, and a surface model is generated based on the at least three initial control points, including:

[0099] Step 401, generate an initial plane model according to the at least three initial control points.

[0100] In this embodiment, taking the number of initial control points as three as an example, the computer device can obtain the spatial coordinates of the three initial control points. For example, the spatial coordinates of the three initial control points A, B, and C are A(x0, y0, z0), B(x1, y1, z1), and C(x2, y2, z2) respectively. The computer device inputs the spatial coordinates A(x0, y0, z0), B(x1, y1, z1), and C(x2, y2, z2) of A, B, and C into the implicit function of the preset plane model for modeling to obtain the initial plane model corresponding to the initial control points. Among them, the preset plane model can be a parallelogram model or a triangle model.

[0101] Step 402, perform rotation and translation operations on the initial plane model according to the preset rotation direction, rotation displacement, and the resolution of the initial plane model, and obtain the target control points corresponding to the initial plane model during the rotation and translation operations.

[0102] In this embodiment, after obtaining the initial plane model, the computer device takes A as the origin of the initial plane model, A, B, and C as the vertices of the initial plane model, and the included angle between the vectors and is used to represent the size of the initial plane model. AB and BC are used as the side lengths of the initial plane model and the X and Y axes of the coordinate system where the initial plane model is located, respectively. The computer device can determine its corresponding centroid point Q through three initial control points A, B, and C, and by adjusting the centroid point Q, perform operations of rotating and translating the initial plane model in the rotation direction and rotation displacement. Among them, the spatial coordinates of the centroid point Q are Q(x m , y m , z m ), where x m is the average value of the X-axis coordinates of A, B, and C; y m is the average value of the Y-axis coordinates of A, B, and C; z m is the average value of the Z-axis coordinates of A, B, and C. Among them, the plane graph formed by A, B, and C can be referred to Figure 5 as shown. The schematic position of point Q is also given in the figure. Taking a quadrilateral as an example, based on A, B, and C, point Q is determined, and then a coordinate system is established with Q as the coordinate origin for translation and rotation.

[0103] Specifically, during the rotation process, the computer device takes Q as the origin of the rectangular coordinate system corresponding to the translation and rotation operations of the initial plane model, and calculates the normal vector and of the initial plane model according to the vectors Arbitrarily select two direction vectors within the initial plane model to satisfy: Taking as the X, Y, and Z directions of the rectangular coordinate system respectively, the computer device translates the initial plane model by a certain displacement along the X or Y or Z direction according to the preset translation step until the initial plane model reaches the boundary of the area to be processed; the computer device rotates the initial plane model by a certain displacement around the X or Y or Z direction according to the preset rotation step until the initial plane model reaches the boundary of the area to be processed.

[0104] After the computer device rotates and translates the initial plane model to form a cutting plane model and performs the above rotation and translation processing, it obtains the cutting plane model after rotation and translation processing, determines the resolution size of the plane model based on the cutting plane model, and thus determines the target control points. Among them, the method for determining the resolution of the plane model includes determining n - 1 equally spaced intervals in the X and Y directions of the plane model respectively, so as to determine an n * n grid. After the computer device determines the grid, it determines each vertex in the grid as a target control point. For example, the computer device determines 3 equally spaced intervals in the X and Y directions respectively, generates a 4 * 4 grid, and determines 16 vertices in the 4 * 4 grid as target control points. Of course, the resolution can be n * n, where n can be a positive integer greater than 1 and less than M such as 2, 3, 4, 5, etc., and the number of corresponding candidate control points is n 2 , where M can be determined based on the volume of the surface model.

[0105] Step 403, generate a surface model according to the target control points and the surface reconstruction algorithm.

[0106] In this embodiment, after the computer device determines the target control points, it inputs the target control points into the implicit function of the surface model and generates a surface model based on the surface reconstruction algorithm. Optionally, the surface reconstruction algorithm in this embodiment can be the Bezier surface reconstruction algorithm.

[0107] In this embodiment, the computer device can generate a surface model corresponding to the area to be processed based on the target control points in the area to be processed in the reconstructed image. The surface model can intuitively reflect the spatial position of the area to be processed, thereby improving the accuracy of the simulated resection based on the surface model.

[0108] During the process of constructing the surface model corresponding to the lesion area, the surface model can also be intuitively adjusted to obtain a more accurate surface model. In one optional embodiment, as Figure 6 shown, the method further includes:

[0109] Step 501, obtain the distance between each voxel point of the surface model and the corresponding voxel point in the area to be resected.

[0110] In this embodiment, after the computer device constructs the surface model, the computer device can obtain the distance between each voxel point in the surface model and the corresponding voxel point in the region to be excised in the reconstructed image according to the correspondence between the surface model and the reconstructed image. For example, the computer device obtains the voxel point W in the surface model, obtains the voxel point W' corresponding to the voxel point W in the region to be excised, and based on the coordinates of the voxel point W and the coordinates of the voxel point W', the computer device determines the distance between these two pixel points, where the voxel point W' is the pixel point corresponding to the position with the shortest spatial distance from the voxel point W to the region to be excised. Optionally, the computer device can calculate the Euclidean distance, spatial distance, or mapping distance mapped on a certain plane between these two pixel points, and this embodiment does not limit this. Taking the organ as the liver and the lesion as the liver tumor as an example, for reference Figure 7 as shown, Figure 7 Figure Figure 7 shows a schematic diagram of the initial plane corresponding to a liver tumor surface model. After constructing the initial plane of the liver tumor surface model, the liver tumor has not been excised yet. Therefore, Figure 7 the liver volume in Figure 7 is still the original volume. For example, the liver volume is 1281.437 cm 3 ³, the excised liver volume is 0 cm 3 ³, and at this time, the residual liver volume (remaining liver volume) is 1281.437 cm 3 ³, and the residual liver ratio is 100%. In Figure 7 the three right-side figures in Figure 7 show schematic diagrams of the initial plane from different perspectives.

[0111] Step 502, if the distance is outside the preset distance threshold range, adjust the surface model to obtain the adjusted surface model.

[0112] In this embodiment, if the computer device determines that the distance between the voxel point in the surface model and the corresponding voxel point in the region to be excised in the reconstructed image is outside the preset distance threshold range, it determines that the surface model is not in the optimal shape. In this case, the computer device can adjust the surface model so that the surface model fits the lesion area better. Among them, the preset distance range can be [5 mm, 20 mm]. Optionally, the computer device can adjust the surface model by adjusting the positions of the voxel points in the surface model; or adjust the surface model based on the plane formed by several voxel points in the surface model. Taking the organ as the liver and the lesion as the liver tumor as an example, the schematic diagram of the surface adjustment by the computer device can be referred to Figure 8 as shown, Figure 8 Figure Figure 8 shows a schematic diagram of the surface model during or after adjusting the surface model through control points. During or after adjusting the surface model through control points, the liver tumor has not been excised yet. Therefore, Figure 8The liver volume in the middle is still the original volume. For example, the liver volume is 1281.437 cm 3 , and the resected liver volume is 0 cm 3 . At this time, the remnant liver volume (remaining liver volume) is 1281.437 cm 3 , and the remnant liver ratio is 100%. In Figure 8 , the left figure includes a surface model, the adjusted plane, and the control points in the plane. The right three figures show the schematic diagrams of the adjusted plane from different perspectives.

[0113] Step 503: Simulate the resection of the lesion based on the adjusted surface model.

[0114] In this embodiment, after obtaining the adjusted surface model, the computer device simulates the resection of the lesion area based on the adjusted surface model. Optionally, the computer device can simulate the resection of the lesion area based on the simulation resection software to obtain the resected area and the remaining area in the organ. Taking the organ as the liver and the lesion as the liver tumor as an example, the schematic diagram of the simulation resection can be referred to Figure 9 as shown, Figure 9 and a schematic diagram after a liver tumor resection operation is given. Figure 9 In 3 , the liver volume is 1281.437 cm 3 , the resected liver volume is 296.608 cm 3 , at this time, the remnant liver volume (remaining liver volume) is 984.829 cm Figure 9 , and the left figure in

[0115] includes the surface model (resected area) and the control points of the plane corresponding to the surface model. The right three figures show the schematic diagrams of the resected area from different perspectives. This embodiment does not limit this.

[0116] In this embodiment, the computer device can adjust the surface model based on the pixel points of the surface model to make the surface model fit the position of the lesion area better, thereby improving the accuracy of the simulation resection based on the surface model. Figure 10 As shown in

[0117] , when adjusting the surface model to obtain the adjusted surface model, it includes:

[0118] Step 601: Adjust the position of the target control point so that the distance between the target control point and the corresponding voxel point in the area to be resected is within the distance threshold range.In this embodiment, the computer device can obtain the coordinates of the target control points, calculate the distances between each target control point and the voxel points of its corresponding region to be excised. If the distance between a target control point and the voxel points of its corresponding region to be excised is outside the distance threshold range, the position of the target control point is adjusted according to a certain adjustment step size, so that the distance between the target control point and the voxel points of its corresponding region to be excised is within the distance threshold range, thereby obtaining the target control point after the position adjustment. For example, in the three-dimensional coordinate system where the surface model is located, the position of the target control point is adjusted according to a preset adjustment step size, so that the distance between the target control point and the voxel points of its corresponding region to be excised is within the distance threshold range, where the preset adjustment step size can be 0.1mm, 0.5mm, 1mm, etc.

[0119] Step 602, generate an adjusted surface model according to the adjusted target control points and the surface reconstruction algorithm.

[0120] In this embodiment, the computer device inputs the adjusted target control points into the surface construction model to obtain an adjusted surface model. Exemplarily, the computer device inputs the adjusted target control points into the implicit function of the surface model for modeling to generate a surface model. Among them, the surface reconstruction technology can be Bezier surface reconstruction technology, and this embodiment does not limit this.

[0121] In this embodiment, the computer device adjusts the distance between the target control point and the voxel points of its corresponding region to be excised by adjusting the position of the target control point, so that the distance between the target control point and the voxel points of its corresponding region to be excised is within the distance threshold range, making the surface model corresponding to the lesion area more accurate.

[0122] Further, the computer device can refine the adjustment granularity. In one optional embodiment, as Figure 11 shown, adjust the surface model to obtain an adjusted surface model, including:

[0123] Step 701, determine a target area within a preset radius range in the surface model based on the voxel points; the target area contains voxel points.

[0124] In this embodiment, the computer device can determine a certain pixel point in the surface model, and determine the target area within the preset radius range based on this pixel point. Exemplarily, the preset radius can be 5mm, 10mm, 20mm, and the range of the preset radius can be [5mm, 20mm]. Thus, the computer device determines the target area based on this pixel point and the preset radius range.

[0125] Step 702, obtain multiple candidate control points of the target area according to the resolution of the target area.

[0126] In this embodiment, after the computer device determines the target area, according to the preset resolution, it determines the resolution grid of the target area. For example, if the preset resolution is 4*4, then a 4*4 grid is generated in the target area. Based on the 4*4 grid, 16 vertices in the grid are determined as candidate control points. Of course, the resolution can be n*n, where n can be a positive integer greater than 1 and less than M such as 2, 3, 4, 5, etc., and the number of corresponding candidate control points is n 2 , where M can be determined based on the volume of the surface model.

[0127] Step 703: Adjust the positions of the candidate control points so that the distances between each candidate control point and the corresponding voxel points in the area to be processed are within the distance threshold range.

[0128] In this embodiment, after determining the candidate control points, the computer device obtains the coordinates of each candidate control point and determines the distance between it and the corresponding voxel point in the area to be processed. Similar to step 601 above, the computer device can adjust the candidate control points in a certain adjustment direction and adjustment step. For example, in the three-dimensional coordinate system where the surface model is located, in the direction of a certain dimension, according to the preset adjustment step, to make the distance between the candidate control point and the voxel point of the corresponding area to be excised within the distance threshold range, where the preset adjustment step can be 0.1mm, 0.5mm, 1mm, etc.

[0129] Step 704: Obtain the adjusted surface model according to the adjusted multiple candidate control points and the surface reconstruction algorithm.

[0130] In this embodiment, the computer device inputs the adjusted candidate control points into the surface construction model to obtain the adjusted surface model. Exemplarily, the computer device inputs the adjusted candidate control points into the implicit function of the surface model for modeling to generate the surface model. Among them, the surface reconstruction technology can be the Bezier surface reconstruction algorithm, which is not limited in this embodiment.

[0131] In this embodiment, the computer device can also further locally adjust the surface model by selecting the target area within the surface, based on the candidate control points in the target area, so that the obtained surface model corresponding to the lesion area is more accurate.

[0132] In one alternative embodiment, the medical image data includes venous phase image data and arterial phase image data; the medical image data is respectively input into different segmentation models to obtain the segmentation results corresponding to each segmentation model, including the following branches:

[0133] Branch 1: Input the arterial phase image data into the first vascular segmentation model to obtain the arterial vascular segmentation result.

[0134] In this embodiment, the first vascular segmentation model can be an arterial segmentation model. The computer device takes the arterial phase image data as the input data of the arterial segmentation model and obtains the arterial vascular segmentation result corresponding to the arterial phase image data based on the arterial segmentation model. Optionally, before inputting the arterial phase image data into the arterial segmentation model, the computer device can also perform data preprocessing on the arterial phase image data. Exemplarily, the data preprocessing includes processing such as image resolution normalization and image gray-scale normalization. Then, the preprocessed arterial phase image data is input into the arterial segmentation model to obtain the arterial vascular segmentation result.

[0135] Branch 2: Input the venous phase image data into the second vascular segmentation model to obtain the venous vascular segmentation result.

[0136] In this embodiment, the second vascular segmentation model can be a venous segmentation model. Further, taking the organ as the liver as an example, the venous phase image data here can be the hepatic portal venous phase image data, and the second vascular segmentation model can be the hepatic portal vein segmentation model. The computer device takes the hepatic portal venous phase image data as the input data of the hepatic portal vein segmentation model and obtains the hepatic vein vascular segmentation result and the portal vein vascular segmentation result corresponding to the hepatic portal venous phase image data based on the hepatic portal vein segmentation model. Optionally, before inputting the hepatic portal venous phase image data into the hepatic portal vein segmentation model, the computer device can also perform data preprocessing on the hepatic portal venous phase image data. Exemplarily, the data preprocessing includes processing such as image resolution normalization and image gray-scale normalization. Then, the preprocessed hepatic portal venous phase image data is input into the hepatic portal vein segmentation model to obtain the hepatic vein vascular segmentation result and the portal vein vascular segmentation result.

[0137] Branch 3 and Branch 4: Input the venous phase image data and the venous vascular segmentation result into the organ segmentation model to obtain the organ segmentation result.

[0138] In this embodiment, taking the organ as the liver and the venous phase image data as the hepatic portal venous phase image data as an example, the organ segmentation model can be a liver segmentation model, a liver segment segmentation model, etc. with different segmentation granularities, and specifically includes the embodiments of Branch 3 and Branch 4 below.

[0139] Optionally, the organ segmentation model includes a first organ segmentation sub-model and a second organ segmentation sub-model; inputting the venous phase image data and the venous vascular segmentation result into the organ segmentation model to obtain the organ segmentation result includes:

[0140] Among them, Branch Three: Input the venous phase image data into the first organ segmentation sub-model to obtain the segmentation result of the first organ.

[0141] In this embodiment, the first organ segmentation sub-model can be a liver segmentation model. Correspondingly, the segmentation result of the first organ is the liver segmentation result. Exemplarily, the computer device uses the hepatic portal venous phase image data as the input data of the liver segmentation model and obtains the liver segmentation result corresponding to the hepatic portal venous phase image data based on the liver segmentation model. Optionally, before inputting the hepatic portal venous phase image data into the liver segmentation model, the computer device can also perform data preprocessing on the hepatic portal venous phase image data. Exemplarily, the data preprocessing includes processing such as image resolution normalization and image gray level normalization. Then, the preprocessed hepatic portal venous phase image data is input into the liver segmentation model to obtain the liver segmentation result.

[0142] Among them, Branch Four: Input the venous blood vessel segmentation result, the segmentation result of the first organ, and the venous phase image data into the second organ segmentation sub-model to obtain the segmentation result of the second organ; the second organ is included in the first organ.

[0143] In this embodiment, the second organ segmentation sub-model can be a liver segment segmentation model for refining the segmentation granularity. Correspondingly, the segmentation result of the second organ is the liver segment segmentation result, where the liver includes multiple liver segments. Exemplarily, the computer device uses the hepatic portal venous phase image data, the hepatic portal venous blood vessel segmentation result obtained in the above embodiment, and the liver segmentation result as the input data of the liver segment segmentation model and obtains the corresponding liver segment segmentation result based on the liver segment segmentation model. Optionally, before inputting the hepatic portal venous phase image data, the hepatic portal venous blood vessel segmentation result obtained in the above embodiment, and the liver segmentation result into the liver segment segmentation model, the computer device can also perform data preprocessing on the hepatic portal venous phase image data, the hepatic portal venous blood vessel segmentation result, and the liver segmentation result. Exemplarily, the data preprocessing includes processing such as image resolution normalization and image gray level normalization. Then, the preprocessed hepatic portal venous phase image data, the hepatic portal venous blood vessel segmentation result, and the liver segmentation result are input into the liver segment segmentation model to obtain the liver segment segmentation result.

[0144] Branch Five: Input the arterial phase image data and the venous phase image data into the lesion segmentation model to obtain the lesion segmentation result.

[0145] In this embodiment, taking the organ as the liver and the lesion as a liver tumor as an example, the lesion segmentation model can be a liver tumor segmentation model. Correspondingly, the lesion segmentation result is the liver tumor segmentation result. Exemplarily, the computer device uses the arterial phase image data and the venous phase image data as the input data of the liver tumor segmentation model, and obtains the corresponding liver tumor segmentation result based on the liver tumor segmentation model. Optionally, before inputting the arterial phase image data and the venous phase image data into the liver tumor segmentation model, the computer device can also perform registration processing on the arterial phase image data and the venous phase image data. Exemplarily, the registration processing includes respectively inputting the arterial phase image data and the venous phase image data into the liver segmentation model to obtain the liver segmentation result of the venous phase data and the liver segmentation result of the arterial phase data. Secondly, the liver regions of the original images corresponding to the arterial phase image data and the liver regions of the original images corresponding to the venous phase image data are respectively cropped by using the obtained liver segmentation results. The liver segmentation result of the cropped venous phase image data is used as the reference image, and the liver segmentation result of the cropped arterial phase image data is used as the floating image for registration to obtain the transformation matrix between the arterial phase image data and the venous phase image data. The cropped arterial phase image data is mapped onto the transformation matrix to obtain the registered arterial phase image data. Finally, the cropped venous phase image data and the registered arterial phase image data are obtained, so that the registered arterial phase image data and the cropped venous phase image data after the registration processing are input into the liver tumor segmentation model to obtain the liver tumor segmentation result. Optionally, before inputting the registered arterial phase image data and the cropped venous phase image data into the liver tumor segmentation model, the computer device can also perform data preprocessing on the registered arterial phase image data and the cropped venous phase image data. Exemplarily, the data preprocessing includes processing such as image resolution normalization and image gray normalization, so that the registered arterial phase image data and the cropped venous phase image data after the data preprocessing are input into the liver tumor segmentation model to obtain the liver tumor segmentation result.

[0146] In addition, since the input data of the liver tumor segmentation model (i.e., the registered arterial phase image data and the cropped venous phase image data) undergoes a cropping operation during the registration process, the output result of the liver tumor segmentation model (the liver tumor segmentation result) is consistent with the size of the cropped input data. However, in the subsequent 3D reconstruction process, the above-mentioned organ segmentation result and blood vessel segmentation result are both in the size before cropping. To ensure that the data sizes involved in the 3D reconstruction are consistent, it is necessary to perform resampling processing on the liver tumor segmentation result here, so that the liver tumor segmentation result is resampled to the size of the image data before cropping, so that the sizes are consistent when performing 3D reconstruction with the above-mentioned organ segmentation result and blood vessel segmentation result.

[0147] Based on the embodiments given in Branch 1, Branch 2, Branch 3, Branch 4, and Branch 5 above, the computer device obtains the arterial vessel segmentation result, the venous vessel segmentation result, the segmentation result of the first organ (liver segmentation result), the segmentation result of the second organ (hepatic segment segmentation result), and the liver tumor segmentation result. The process of organ, vessel, and lesion segmentation based on each segmentation model can be referred to Figure 12 as shown. It should be noted that the segmentation models involved in the above embodiments can be neural network models constructed based on the VBNet neural network framework, or other neural network models. These neural network models are all segmentation models obtained after training and adapted to each segmentation scenario. This embodiment does not make any limitations in this regard.

[0148] In this embodiment, based on multiple preset deep learning segmentation models, the segmentation results corresponding to the organs, vessels, and lesions are obtained. The obtained segmentation results are relatively accurate, providing effective data support for the next step of 3D reconstruction based on the segmentation results.

[0149] The above-mentioned segmentation results corresponding to the venous phase image data are obtained based on the venous phase image data and each segmentation model, and the segmentation results corresponding to the arterial phase image data are obtained based on the arterial phase image data and each segmentation model. Before performing 3D reconstruction based on each segmentation result, it is necessary to register the venous phase image data and the arterial phase image data. In one optional embodiment, as Figure 13 shown, 3D reconstruction processing is performed according to each segmentation result to obtain a reconstructed image, including:

[0150] Step 801, register the arterial vessel segmentation result based on the venous phase image data, the arterial phase image data, and the image registration algorithm to obtain the registered arterial vessel segmentation result.

[0151] Among them, the image registration algorithm can be relative registration or absolute registration. Exemplarily, the relative registration algorithm refers to using the venous phase image data as the reference image, the arterial phase image data as the floating image, and registering the arterial phase image data with the venous phase image data. The coordinate system involved in the registration process can be the coordinate system corresponding to the venous phase image data. Absolute registration refers to first defining a control grid, registering both the venous phase image data and the arterial phase image data with the control grid, and finally forming a registration result that is the same as the coordinate system of the control grid.

[0152] In the specific registration process, a registration function mapping relationship between venous phase image data and arterial phase image data is required. Exemplarily, a computer device can fit the translation, rotation, and affine transformation between the venous phase image data and the arterial phase image data by setting a polynomial, and thus determine the image registration mapping relationship between the venous phase image data and the arterial phase image data by calculating the coefficients of the polynomial. Exemplarily, the venous phase image data can be used as the reference image, and the arterial phase image data can be used as the floating image to determine the image registration mapping relationship between the venous phase image data and the arterial phase image data. According to the image registration mapping relationship, the mapping of the arterial vessel segmentation result corresponding to the arterial phase image data to the venous phase image data is realized, so as to obtain the arterial vessel segmentation result after registration. This embodiment does not make any limitation on this.

[0153] Step 802: Perform three-dimensional reconstruction based on the arterial vessel segmentation result, venous vessel segmentation result, lesion segmentation result, and segmentation result of the second organ after registration to obtain a reconstructed image.

[0154] In this embodiment, after the computer device obtains the arterial vessel segmentation result corresponding to the arterial phase image data after registration, the arterial vessel segmentation result after registration, the venous vessel segmentation result, the lesion segmentation result, and the segmentation result of the second organ are all segmentation results in the reference system where the venous phase image data is located. Based on the same reference system, three-dimensional reconstruction processing is performed according to the arterial vessel segmentation result, venous vessel segmentation result, lesion segmentation result, and segmentation result of the second organ to obtain a three-dimensional visualization image including the spatial relationship among the organ, blood vessels, and lesions. Taking the organ as the liver as an example, the obtained reconstructed image includes the spatial position relationship among the liver, blood vessels, and liver tumors. In order to more intuitively display the liver, blood vessels, and liver tumors in the reconstructed image, the schematic diagram provided in this embodiment visually disassembles the liver, blood vessels, and liver tumors in the reconstructed image (in fact, the spatial relationship among the three parts is superimposed on each other). For details, reference can be made to Figure 14 , Figure 14 which gives the disassembled parts corresponding to the liver, blood vessels, and liver tumors in the reconstructed image. This embodiment does not make any limitation on this.

[0155] In this embodiment, the computer device performs registration on the segmentation results corresponding to the arterial phase image data according to the venous phase image data and the arterial phase image data, so as to obtain each segmentation result converted to the same registration matrix. The three-dimensional visualization reconstructed image obtained by performing three-dimensional reconstruction based on each segmentation result after registration is more accurate, that is, it can more accurately express the orientation relationship and spatial position relationship among the organ, blood vessels, and lesions.

[0156] One specific implementation method for registering the venous phase image data and the arterial phase image data in step 801 above can be achieved through a transformation matrix. In one alternative embodiment, as Figure 15 shown, based on the venous phase image data, the arterial phase image data, and an image registration algorithm, the arterial vessel segmentation result is registered to obtain the registered arterial vessel segmentation result, including:

[0157] Step 901: Use the venous phase image data as the reference image and the arterial phase image data as the floating image to determine the registration transformation matrix between the venous phase image data and the arterial phase image data.

[0158] In this embodiment, the computer device uses the arterial phase image data as the floating image (moving image) and the venous phase image data as the reference image (fixed image), and extracts the features in the floating image and the reference image respectively. For example, the features may include closed boundaries, edges, contours, line intersections, corner points, and their representative points (such as control points like the centroid or the end of a line, etc.). After feature extraction, the computer device establishes the correspondence between the features of the floating image and the features of the reference image based on the features of the floating image and the features of the reference image, and determines the registration transformation matrix between the venous phase image data and the arterial phase image data based on the correspondence between the features of the floating image and the features of the reference image. Exemplarily, the registration transformation matrix can also be understood as a mapping function. Optionally, the computer device can calculate the parameters involved in the mapping function based on the correspondence between the features of the floating image and the features of the reference image, so as to determine the final mapping function, that is, the registration transformation matrix.

[0159] Step 902: Map the arterial vessel segmentation result into the registration transformation matrix to obtain the registered arterial vessel segmentation result.

[0160] In this embodiment, after the computer device determines the registration transformation matrix between the venous phase image data and the arterial phase image data, it maps the arterial vessel segmentation result corresponding to the floating image, that is, the arterial phase image data, into the registration transformation matrix, so as to obtain the arterial vessel segmentation result registered with the venous phase image data.

[0161] In this embodiment, the computer device uses the venous phase image data as the reference image and the arterial phase image data as the floating image to register the segmentation result corresponding to the arterial phase image data, so as to obtain each segmentation result transformed to the same registration matrix. The three-dimensional visualization reconstruction image obtained by three-dimensional reconstruction based on each segmentation result is more accurate.

[0162] After determining the surface model corresponding to the lesion area, a simulated resection can be performed based on the surface model to obtain a simulated resection plan for the lesion. In one alternative embodiment, as Figure 16 shown, the method further includes:

[0163] Step 1001, perform a simulated resection on the lesion based on the surface model to obtain the resected area and the remaining area.

[0164] In this embodiment, the computer device can perform a simulated resection on the lesion based on the surface model through simulation software. Exemplarily, the computer device can use a surface simulation scalpel in the simulation software to excise the lesion area to be cut in the surface model to obtain the resected area and the remaining area. Among them, taking the resection of a liver tumor as an example, the resected area includes the liver segment area where the liver tumor is located, and the remaining area is the liver segment area after removing the liver segment area where the liver tumor is located.

[0165] Step 1002, determine the quantitative information of the organ according to the resected area and the remaining area; the quantitative information is used to represent the volume ratio of the remaining area to the overall area of the organ.

[0166] In this embodiment, after the computer device performs the resection operation, it obtains the resected area and the remaining area. It can determine multiple candidate three-dimensional coordinates of the resected area based on a preset three-dimensional coordinate axis, and calculate the volume of the resected area based on the multiple candidate three-dimensional coordinates of the resected area; determine multiple candidate three-dimensional coordinates of the remaining area based on a preset three-dimensional coordinate axis, and calculate the volume of the remaining area based on the multiple candidate three-dimensional coordinates of the remaining area, so as to determine the volume ratio of the remaining area to the overall area of the organ according to the volume of the resected area and the volume of the remaining area. Optionally, before performing the resection operation, the computer device can first calculate the volume of the entire organ. Similarly, it can determine multiple candidate three-dimensional coordinates of the organ based on a preset three-dimensional coordinate axis, and determine the volume of the organ based on the multiple candidate three-dimensional coordinates of the organ. Taking the resection of a liver tumor and the organ being the liver as an example, the resected area is the liver segment area where the liver tumor is located, and the remaining area is the remaining liver segment area. The computer device determines the overall volume of the liver, the volume of the liver segment where the liver tumor is located, and the volume of the remaining liver segment based on the above method, so as to determine the ratio of the volume of the remaining liver segment to the overall volume of the liver (residual liver ratio) as the quantitative information of the liver. The specific resection surface can be referred to Figure 9 shown.

[0167] In this embodiment, the computer device performs a more accurate simulated resection of the lesion based on the surface model. The simulated resection plan obtained based on the simulated resection can provide effective data support for the user's lesion resection operation, and further improve the efficiency and accuracy of the lesion resection.

[0168] To better illustrate the above method, asFigure 17 As shown in the figure, this embodiment provides a method for processing image data, which specifically includes:

[0169] S101. Input the arterial phase image data in the medical image data into the first blood vessel segmentation model to obtain the arterial blood vessel segmentation result;

[0170] S102. Input the venous phase image data in the medical image data into the second blood vessel segmentation model to obtain the venous blood vessel segmentation result;

[0171] S103. Input the venous phase image data into the first organ segmentation sub-model to obtain the segmentation result of the first organ;

[0172] S104. Input the venous blood vessel segmentation result, the segmentation result of the first organ, and the venous phase image data into the second organ segmentation sub-model to obtain the segmentation result of the second organ;

[0173] S105. Input the arterial phase image data and the venous phase image data into the lesion segmentation model to obtain the lesion segmentation result;

[0174] S106. Based on the venous phase image data, the arterial phase image data, and the image registration algorithm, register the arterial blood vessel segmentation result to obtain the registered arterial blood vessel segmentation result;

[0175] S107. Perform three-dimensional reconstruction according to the registered arterial blood vessel segmentation result, the venous blood vessel segmentation result, the lesion segmentation result, and the segmentation result of the second organ to obtain a reconstructed image;

[0176] S108. Based on the position of the lesion in the reconstructed image, determine the area to be processed corresponding to the lesion on the organ;

[0177] S109. Set at least three initial control points in the area to be processed, and generate a surface model based on the at least three initial control points;

[0178] S110. If the distance is outside the preset distance threshold range, adjust the surface model to obtain the adjusted surface model;

[0179] S111. Perform simulated resection of the lesion based on the adjusted surface model to obtain the resected area and the remaining area;

[0180] S112. Determine the quantitative information of the organ according to the resected area and the remaining area.

[0181] In this embodiment, in this solution, the computer device first obtains the segmentation results corresponding to each segmentation model based on the segmentation models corresponding to the organ, blood vessel, and lesion respectively. The reconstructed images obtained based on each segmentation result can better reflect the relationship between the organ, blood vessel, and lesion, eliminating the need for the user to manually extract features for 3D reconstruction and reducing the time for 3D reconstruction. A surface model of the lesion area is constructed based on the reconstructed images. A plane is established on the 3D visualization surface using control points, and then surface control points are generated using the plane to form a preliminary adjustment surface. The position, direction, size, etc. of the adjustment surface are determined. A certain area inside the surface is selected to perform precise local adjustment on the surface, and the cutting area is set more accurately, so that the simulated resection of the lesion based on the surface model is more accurate. The simulated resection plan obtained based on the simulated resection can provide effective data support for the user's lesion resection operation, further improving the efficiency and accuracy of lesion resection.

[0182] The imaging data processing method provided in the above embodiment has the same implementation principle and technical effects as those in the above method embodiment, and will not be elaborated here.

[0183] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps in other steps.

[0184] Based on the same inventive concept, the embodiments of the present application also provide an imaging data processing device for implementing the above-mentioned imaging data processing method. The solution provided by this device for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the imaging data processing device provided below can refer to the limitations on the imaging data processing method in the above text, and will not be elaborated here.

[0185] In one embodiment, as Figure 18 shown, an imaging data processing device is provided, including:

[0186] A segmentation module 01, configured to input medical imaging data into different segmentation models respectively to obtain the segmentation results corresponding to each segmentation model; the medical imaging data includes an organ, a blood vessel, and a lesion;

[0187] A three-dimensional reconstruction module 02, configured to perform three-dimensional reconstruction based on each segmentation result to obtain a reconstructed image; the reconstructed image includes organs, blood vessels, and lesions.

[0188] A simulation module 03, configured to construct a surface model of the lesion based on the reconstructed image and perform a simulated resection of the lesion based on the surface model.

[0189] In one optional embodiment, the simulation module 03 is configured to determine a to-be-processed area corresponding to the lesion on the organ based on the position of the lesion in the reconstructed image; set at least three initial control points in the to-be-processed area, and generate a surface model based on the at least three initial control points.

[0190] In one optional embodiment, the simulation module 03 is configured to generate an initial plane model according to at least three initial control points; perform rotation and translation operations on the initial plane model according to a preset rotation direction, rotation displacement, and the resolution of the initial plane model to obtain target control points corresponding to the initial plane model during the rotation and translation operations; and generate a surface model according to the target control points and a surface reconstruction algorithm.

[0191] In one optional embodiment, the simulation module 03 is further configured to obtain the distance between each voxel point of the surface model and the corresponding voxel point in the to-be-resected area; if the distance is outside a preset distance threshold range, adjust the surface model to obtain an adjusted surface model; and perform a simulated resection of the lesion based on the adjusted surface model.

[0192] In one optional embodiment, the simulation module 03 is configured to adjust the positions of the target control points so that the distances between the target control points and the corresponding voxel points in the to-be-resected area are within the distance threshold range; and generate an adjusted surface model according to the adjusted target control points and a surface reconstruction algorithm.

[0193] In one optional embodiment, the simulation module 03 is configured to determine a target area within a preset radius range in the surface model based on the voxel points; the target area contains the voxel points; obtain a plurality of candidate control points of the target area according to the resolution of the target area; adjust the positions of the candidate control points so that the distances between the candidate control points and the corresponding voxel points in the to-be-processed area are within the distance threshold range; and obtain an adjusted surface model according to the adjusted plurality of candidate control points and a surface reconstruction algorithm.

[0194] In one alternative embodiment, the medical image data includes venous phase image data and arterial phase image data; a segmentation module 01, configured to input the arterial phase image data into a first blood vessel segmentation model to obtain an arterial blood vessel segmentation result; input the venous phase image data into a second blood vessel segmentation model to obtain a venous blood vessel segmentation result; input the venous phase image data and the venous blood vessel segmentation result into an organ segmentation model to obtain an organ segmentation result; and input the arterial phase image data and the venous phase image data into a lesion segmentation model to obtain a lesion segmentation result.

[0195] In one alternative embodiment, the organ segmentation model includes a first organ segmentation sub-model and a second organ segmentation sub-model; the segmentation module 01 is configured to input the venous phase image data into the first organ segmentation sub-model to obtain a segmentation result of the first organ; input the venous blood vessel segmentation result, the segmentation result of the first organ, and the venous phase image data into the second organ segmentation sub-model to obtain a segmentation result of the second organ; and the first organ includes the second organ.

[0196] In one alternative embodiment, a 3D reconstruction module 02 is configured to register the arterial blood vessel segmentation result based on the venous phase image data, the arterial phase image data, and an image registration algorithm to obtain a registered arterial blood vessel segmentation result; perform 3D reconstruction based on the registered arterial blood vessel segmentation result, the venous blood vessel segmentation result, the lesion segmentation result, and the segmentation result of the second organ to obtain a reconstructed image.

[0197] In one alternative embodiment, the 3D reconstruction module 02 is configured to use the venous phase image data as a reference image and the arterial phase image data as a floating image to determine a registration transformation matrix between the venous phase image data and the arterial phase image data; and map the arterial blood vessel segmentation result into the registration transformation matrix to obtain a registered arterial blood vessel segmentation result.

[0198] In one alternative embodiment, as Figure 19 shown, the device further includes a quantification module 04;

[0199] The quantification module 04 is configured to simulate the resection of the lesion based on a surface model to obtain a resected area and a remaining area; determine quantification information of the organ according to the resected area and the remaining area; and the quantification information is used to represent the volume ratio of the remaining area to the overall area of the organ.

[0200] Each module in the above image data processing device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above respective modules.

[0201] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0202] Input medical image data into different segmentation models respectively to obtain segmentation results corresponding to the respective segmentation models; the medical image data includes organs, blood vessels, and lesions;

[0203] Perform three-dimensional reconstruction based on the respective segmentation results to obtain a reconstructed image; the reconstructed image includes organs, blood vessels, and lesions;

[0204] Construct a surface model of the lesion based on the reconstructed image, and perform simulated resection of the lesion based on the surface model.

[0205] For the computer device provided in the above embodiment, its implementation principle and technical effects are similar to those of the above method embodiment, and will not be elaborated here.

[0206] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0207] Input medical image data into different segmentation models respectively to obtain segmentation results corresponding to the respective segmentation models; the medical image data includes organs, blood vessels, and lesions;

[0208] Perform three-dimensional reconstruction based on the respective segmentation results to obtain a reconstructed image; the reconstructed image includes organs, blood vessels, and lesions;

[0209] Construct a surface model of the lesion based on the reconstructed image, and perform simulated resection of the lesion based on the surface model.

[0210] For the computer-readable storage medium provided in the above embodiment, its implementation principle and technical effects are similar to those of the above method embodiment, and will not be elaborated here.

[0211] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0212] Input medical image data into different segmentation models respectively to obtain segmentation results corresponding to the respective segmentation models; the medical image data includes organs, blood vessels, and lesions;

[0213] Perform three-dimensional reconstruction based on the respective segmentation results to obtain a reconstructed image; the reconstructed image includes organs, blood vessels, and lesions;

[0214] Construct a surface model of the lesion based on the reconstructed image, and perform simulated resection of the lesion based on the surface model.

[0215] The computer program product provided in the above embodiments has the same implementation principle and technical effects as the above method embodiments, and will not be elaborated herein.

[0216] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0217] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0218] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0219] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. An image data processing method, characterized in that, The method includes: Inputting medical image data into different segmentation models respectively to obtain segmentation results corresponding to each of the segmentation models; the medical image data includes organs, blood vessels, and lesions; Performing three-dimensional reconstruction based on each of the segmentation results to obtain a reconstructed image; the reconstructed image includes organs, blood vessels, and lesions; Constructing a surface model of the lesion based on the reconstructed image, and performing simulated resection of the lesion based on the surface model; The constructing the surface model of the lesion based on the reconstructed image includes: Based on the position of the lesion in the reconstructed image, determining a region to be processed corresponding to the lesion on the organ, generating an initial plane model according to at least three initial control points in the region to be processed, performing rotation and translation operations on the initial plane model according to a preset rotation direction, rotation displacement, and the resolution of the initial plane model, obtaining target control points corresponding to the initial plane model during the rotation and translation operations, and generating the surface model according to the target control points and a surface reconstruction algorithm.

2. The method according to claim 1, characterized in that, The method further includes: Obtaining the distance between each voxel point of the surface model and the corresponding voxel point in the region to be resected; If the distance is outside a preset distance threshold range, adjusting the surface model to obtain an adjusted surface model; The performing simulated resection of the lesion based on the surface model includes: Performing simulated resection of the lesion based on the adjusted surface model.

3. The method according to claim 2, wherein The adjusting the surface model to obtain an adjusted surface model includes: Determining a target region within a preset radius range in the surface model based on the voxel point; the target region contains the voxel point; Obtaining a plurality of candidate control points of the target region according to the resolution of the target region; Adjusting the positions of the candidate control points so that the distances between each of the candidate control points and the corresponding voxel points in the region to be processed are within the distance threshold range; Obtaining the adjusted surface model according to the adjusted plurality of candidate control points and a surface reconstruction algorithm.

4. The method according to any one of claims 1 to 3, characterized in that, The medical image data includes venous phase image data and arterial phase image data; the inputting the medical image data into different segmentation models respectively to obtain segmentation results corresponding to each of the segmentation models includes: Inputting the arterial phase image data into a first blood vessel segmentation model to obtain an arterial blood vessel segmentation result; Inputting the venous phase image data into a second blood vessel segmentation model to obtain a venous blood vessel segmentation result; Inputting the venous phase image data and the venous blood vessel segmentation result into an organ segmentation model to obtain an organ segmentation result; Inputting the arterial phase image data and the venous phase image data into a lesion segmentation model to obtain a lesion segmentation result.

5. The method according to claim 4, characterized in that, The organ segmentation model includes a first organ segmentation sub-model and a second organ segmentation sub-model; the inputting the venous phase image data and the venous blood vessel segmentation result into the organ segmentation model to obtain an organ segmentation result includes: Inputting the venous phase image data into the first organ segmentation sub-model to obtain a segmentation result of the first organ; Input the segmentation result of the venous blood vessels, the segmentation result of the first organ, and the venous phase image data into the second organ segmentation sub-model to obtain the segmentation result of the second organ; the second organ is included in the first organ.

6. The method according to claim 5, characterized in that, Performing three-dimensional reconstruction based on each of the segmentation results to obtain a reconstructed image, including: Register the segmentation result of the arterial blood vessels based on the venous phase image data, the arterial phase image data, and an image registration algorithm to obtain the registered segmentation result of the arterial blood vessels; Perform three-dimensional reconstruction based on the registered segmentation result of the arterial blood vessels, the segmentation result of the venous blood vessels, the segmentation result of the lesion, and the segmentation result of the second organ to obtain the reconstructed image.

7. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images