Method for generating lesion boundary on surface of viscera three-dimensional model and electronic equipment

Abstract

The invention discloses a viscera three-dimensional model surface lesion boundary generation method and electronic equipment, and the method comprises the steps: firstly converting a three-dimensional coordinate of a lesion surface grid vertex into a two-dimensional coordinate, and then selecting the two-dimensional coordinate of the lesion surface grid vertex representing the minimum boundary point of a lesion, determining the minimum focus section of the focus based on the two-dimensional coordinates of the selected focus surface grid vertex and the focus center point, calculating the minimum boundary point of the focus according to the minimum focus section, and finally obtaining the safety boundary point of the focus based on the minimum boundary point of the focus and the preset distance. And extending the safety boundary point of each lesion in the corresponding normal vector direction to obtain an intersection point with the surface of the viscera three-dimensional model, and taking the obtained intersection point as the safety boundary point of the lesion on the surface of the viscera three-dimensional model, thereby automatically generating the safety boundary point of the lesion on the surface of the viscera three-dimensional model. Compared with a method for manually marking the boundary of the focus, the method is easy to operate and high in efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for generating a lesion boundary on the surface of a three-dimensional model of an internal organ and an electronic device. Background Art

[0002] Laparoscopic surgery has the advantages of small incision, less pain, quick recovery and small scars, and Kuaishou calls it the current development trend of surgery. In order to make the surgery faster, more accurate and safer, it is usually necessary to align and fusion display the 3D model of the internal organs, so that the internal organs can be viewed more intuitively during the operation.

[0003] However, for tumors located inside the internal organs, the position of the tumor boundary on the surface of the internal organ 3D model cannot be directly displayed on the display interface. In the prior art, the tumor boundary is manually marked on the surface of the internal organ 3D model. The manual method of marking the tumor boundary on the surface of the internal organ 3D model is complicated and inefficient. Summary of the invention

[0004] The present invention provides a method for generating a lesion boundary on the surface of a three-dimensional visceral model and an electronic device, so as to solve the problem in the prior art that the method of manually marking the boundary of a tumor on the surface of a three-dimensional visceral model is complicated in operation and low in efficiency.

[0005] In a first aspect, an embodiment of the present application provides a method for generating a lesion boundary on the surface of a 3D model of an internal organ, the method comprising:

[0006] Converting the three-dimensional coordinates of each lesion surface mesh vertex into two-dimensional coordinates of the lesion surface mesh vertex, wherein the three-dimensional coordinates of each lesion surface mesh vertex are obtained based on the registered three-dimensional lesion model;

[0007] Selecting the two-dimensional coordinates of the vertices on the lesion surface mesh representing the minimum boundary point of the lesion from the two-dimensional coordinates of the vertices on the lesion surface mesh;

[0008] Determine the minimum lesion cross section of the lesion based on the two-dimensional coordinates of the selected lesion surface mesh vertices and the lesion center point, and calculate the minimum boundary point of the lesion on the surface of the lesion three-dimensional model according to the minimum lesion cross section;

[0009] After obtaining the safety boundary points of the lesion based on the minimum boundary point and the preset distance on the surface of the three-dimensional model of the lesion, each safety boundary point of the lesion is extended in the direction of the corresponding normal vector, and the first intersection point with the surface of the three-dimensional model of the viscera is used as the safety boundary point of the lesion on the surface of the three-dimensional model of the viscera and is displayed on the laparoscopic image of the current view.

[0010] In a possible implementation, the step of selecting the two-dimensional coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion from the two-dimensional coordinates of the lesion surface mesh vertex includes:

[0011] Segmenting the model projection of the three-dimensional model of the lesion into a preset number of segmentation areas along the vertical direction;

[0012] According to the two-dimensional coordinates of the lesion surface mesh vertices within the segmented area, the two-dimensional coordinates of the lesion surface mesh vertices representing the minimum boundary point of the lesion are selected.

[0013] In a possible implementation, selecting the two-dimensional coordinates of the lesion surface mesh vertices representing the minimum boundary point of the lesion according to the two-dimensional coordinates of the lesion surface mesh vertices in the segmented area includes:

[0014] Determine, among the two-dimensional coordinates of the vertices on the surface of the lesion mesh within the segmented region, the two-dimensional coordinate with the largest abscissa, the two-dimensional coordinate with the smallest abscissa, the two-dimensional coordinate with the largest ordinate, and the two-dimensional coordinate with the smallest ordinate;

[0015] The determined two-dimensional coordinate with the largest abscissa, the two-dimensional coordinate with the smallest abscissa, the two-dimensional coordinate with the largest ordinate, and the two-dimensional coordinate with the smallest ordinate are used as the two-dimensional coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion.

[0016] In a possible implementation, determining the minimum lesion cross section of the lesion based on the two-dimensional coordinates of the selected lesion surface mesh vertices and the lesion center point includes:

[0017] Calculating a first distance between any two selected two-dimensional coordinates of each lesion surface mesh vertex to obtain a set of two-dimensional coordinates of a group of lesion surface mesh vertices with the largest first distance;

[0018] Taking the line connecting the two-dimensional coordinates of the group of lesion surface mesh vertices with the largest distance as the central axis, calculating the second distance between the two-dimensional coordinates of each lesion surface mesh vertex and the central axis, and determining the two-dimensional coordinates of the two target lesion surface mesh vertices with the largest second distance on both sides of the central axis;

[0019] Converting the two-dimensional coordinates of the vertices of the target lesion surface mesh into three-dimensional coordinates;

[0020] An interface determined according to the three-dimensional coordinates of the two target lesion surface mesh vertices and the three-dimensional coordinates of the lesion center point is used as the minimum lesion cross section of the lesion.

[0021] In a possible implementation, calculating the minimum boundary point of the lesion on the surface of the lesion three-dimensional model according to the minimum lesion cross section includes:

[0022] The intersection point between the minimum lesion cross section of the lesion and the surface of the lesion three-dimensional model is calculated, and the points corresponding to the obtained multiple three-dimensional coordinates are used as the minimum boundary points of the lesion on the surface of the lesion three-dimensional model.

[0023] In a possible implementation, obtaining the safe boundary point of the lesion based on the minimum boundary point and the preset distance on the surface of the three-dimensional model of the lesion includes:

[0024] For each minimum boundary point of the lesion on the surface of the three-dimensional model of the lesion: according to the two-dimensional coordinates of the minimum boundary point, in the direction from the lesion center point to the minimum boundary point, the minimum boundary point is extended by the preset distance to obtain a safe boundary point of the lesion.

[0025] In a possible implementation, extending the safety boundary point of each lesion upward along the corresponding normal vector to obtain a first intersection point with the surface of the visceral three-dimensional model as the safety boundary point of the lesion on the surface of the visceral three-dimensional model includes:

[0026] For each safety boundary point of the lesion: according to the triangular facet to which the safety boundary point of the lesion belongs, the normal vector coordinates corresponding to the safety boundary point of the lesion are calculated; according to the three-dimensional coordinates of the safety boundary point of the lesion and the normal vector coordinates, the ray corresponding to the safety boundary point of the lesion is determined; according to the three-dimensional coordinates of the surface mesh vertices of the three-dimensional model of the internal organs and the ray, the first intersection point of the safety boundary point of the lesion and the surface of the three-dimensional model of the internal organs is obtained;

[0027] The first intersection point obtained is used as a safe boundary point of the lesion on the surface of the three-dimensional model of the internal organs.

[0028] In a possible implementation, before displaying on the laparoscopic image of the current view, the method further includes:

[0029] Converting the safety boundary points of the lesion on the surface of the three-dimensional model of the internal organs into two-dimensional coordinates;

[0030] Plot the transformed 2D coordinates.

[0031] In a possible implementation, after calculating the minimum boundary point of the lesion on the surface of the lesion three-dimensional model according to the minimum lesion cross section, the method further includes:

[0032] Extending the minimum boundary point of each lesion on the surface of the lesion three-dimensional model in the direction of the corresponding normal vector to obtain a second intersection point with the surface of the viscera three-dimensional model;

[0033] The obtained second intersection point is used as the minimum boundary point of the lesion on the surface of the three-dimensional model of the internal organs, and is displayed on the laparoscopic image of the current view.

[0034] In a second aspect, an embodiment of the present application provides an electronic device, the electronic device comprising a processor and a display screen;

[0035] The processor is used to convert the three-dimensional coordinates of each lesion surface mesh vertex of the lesion into the two-dimensional coordinates of the lesion surface mesh vertex, wherein the three-dimensional coordinates of each lesion surface mesh vertex are obtained based on the registered lesion three-dimensional model; select the two-dimensional coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion from the two-dimensional coordinates of the lesion surface mesh vertices; determine the minimum lesion cross section of the lesion based on the selected two-dimensional coordinates of the lesion surface mesh vertex and the lesion center point, and calculate the minimum boundary point of the lesion on the surface of the lesion three-dimensional model according to the minimum lesion cross section; after obtaining the safety boundary point of the lesion based on the minimum boundary point and the preset distance on the surface of the lesion three-dimensional model, extend each safety boundary point of the lesion in the direction of the corresponding normal vector, and obtain the first intersection point with the surface of the visceral three-dimensional model as the safety boundary point of the lesion on the surface of the visceral three-dimensional model;

[0036] The display screen is used to display the safe boundary points of the lesion on the surface of the three-dimensional model of the internal organs on the laparoscopic image of the current view.

[0037] In a possible implementation manner, the processor is specifically configured to:

[0038] Segmenting the model projection of the three-dimensional model of the lesion into a preset number of segmentation areas along the vertical direction;

[0039] According to the two-dimensional coordinates of the lesion surface mesh vertices within the segmented area, the two-dimensional coordinates of the lesion surface mesh vertices representing the minimum boundary point of the lesion are selected.

[0040] In a possible implementation manner, the processor is specifically configured to:

[0041] Determine, among the two-dimensional coordinates of the vertices on the surface of the lesion mesh within the segmented region, the two-dimensional coordinate with the largest abscissa, the two-dimensional coordinate with the smallest abscissa, the two-dimensional coordinate with the largest ordinate, and the two-dimensional coordinate with the smallest ordinate;

[0042] The determined two-dimensional coordinate with the largest abscissa, the two-dimensional coordinate with the smallest abscissa, the two-dimensional coordinate with the largest ordinate, and the two-dimensional coordinate with the smallest ordinate are used as the two-dimensional coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion.

[0043] In a possible implementation manner, the processor is specifically configured to:

[0044] Calculating a first distance between any two selected two-dimensional coordinates of each lesion surface mesh vertex to obtain a set of two-dimensional coordinates of a group of lesion surface mesh vertices with the largest first distance;

[0045] Taking the line connecting the two-dimensional coordinates of the group of lesion surface mesh vertices with the largest distance as the central axis, calculating the second distance between the two-dimensional coordinates of each lesion surface mesh vertex and the central axis, and determining the two-dimensional coordinates of the two target lesion surface mesh vertices with the largest second distance on both sides of the central axis;

[0046] Converting the two-dimensional coordinates of the vertices of the target lesion surface mesh into three-dimensional coordinates;

[0047] An interface determined according to the three-dimensional coordinates of the two target lesion surface mesh vertices and the three-dimensional coordinates of the lesion center point is used as the minimum lesion cross section of the lesion.

[0048] In a possible implementation manner, the processor is specifically configured to:

[0049] The intersection point between the minimum lesion cross section of the lesion and the surface of the lesion three-dimensional model is calculated, and the points corresponding to the obtained multiple three-dimensional coordinates are used as the minimum boundary points of the lesion on the surface of the lesion three-dimensional model.

[0050] In a possible implementation manner, the processor is specifically configured to:

[0051] For each minimum boundary point of the lesion on the surface of the three-dimensional model of the lesion: according to the two-dimensional coordinates of the minimum boundary point, in the direction from the lesion center point to the minimum boundary point, the minimum boundary point is extended by the preset distance to obtain a safe boundary point of the lesion.

[0052] In a possible implementation manner, the processor is specifically configured to:

[0053] For each safety boundary point of the lesion: according to the triangular facet to which the safety boundary point of the lesion belongs, the normal vector coordinates corresponding to the safety boundary point of the lesion are calculated; according to the three-dimensional coordinates of the safety boundary point of the lesion and the normal vector coordinates, the ray corresponding to the safety boundary point of the lesion is determined; according to the three-dimensional coordinates of the surface mesh vertices of the three-dimensional model of the internal organs and the ray, the first intersection point of the safety boundary point of the lesion and the surface of the three-dimensional model of the internal organs is obtained;

[0054] The first intersection point obtained is used as a safe boundary point of the lesion on the surface of the three-dimensional model of the internal organs.

[0055] In a possible implementation, before displaying on the laparoscopic image of the current view, the processor is further configured to:

[0056] Converting the safety boundary points of the lesion on the surface of the three-dimensional model of the internal organs into two-dimensional coordinates;

[0057] Plot the transformed 2D coordinates.

[0058] In a possible implementation manner, the processor is further configured to:

[0059] Extending the minimum boundary point of each lesion on the surface of the lesion three-dimensional model in the direction of the corresponding normal vector to obtain a second intersection point with the surface of the viscera three-dimensional model;

[0060] The obtained second intersection point is used as the minimum boundary point of the lesion on the surface of the three-dimensional model of the internal organs, and is displayed on the laparoscopic image of the current view.

[0061] The beneficial effects of the present invention are as follows:

[0062] The embodiment of the present application provides a method for generating the boundary of a lesion on the surface of a three-dimensional visceral model and an electronic device. The method comprises: firstly converting the three-dimensional coordinates of each lesion surface mesh vertex in the registered three-dimensional lesion model into two-dimensional coordinates of the lesion surface mesh vertex; then selecting the two-dimensional coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion from the two-dimensional coordinates of the lesion surface mesh vertex; determining the minimum lesion cross section of the lesion based on the selected two-dimensional coordinates of the lesion surface mesh vertex and the lesion center point; and calculating the minimum boundary point of the lesion on the surface of the three-dimensional lesion model according to the minimum lesion cross section; and finally, obtaining the safety boundary point of the lesion based on the minimum boundary point and a preset distance on the surface of the three-dimensional lesion model; extending the safety boundary point of each lesion in the direction of the corresponding normal vector; and obtaining the first intersection point with the surface of the three-dimensional visceral model as the safety boundary point of the lesion on the surface of the three-dimensional visceral model; and displaying the first intersection point with the surface of the three-dimensional visceral model on the laparoscopic image of the current view, thereby realizing automatic generation of the safety boundary point of the lesion on the surface of the three-dimensional visceral model. Compared with the method of manually marking the boundary of the lesion on the surface of the three-dimensional visceral model, the method is simple to operate and has high efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0064] Figure 1A schematic diagram of the structure of a laparoscopic augmented reality surgical navigation system provided in an embodiment of the present application;

[0065] Figure 2 A schematic flow chart of a method for generating a lesion boundary on the surface of a 3D model of an internal organ provided in an embodiment of the present application;

[0066] Figure 3 A schematic diagram of a three-dimensional model of a liver obtained by three-dimensional modeling using Vtk provided in an embodiment of the present application;

[0067] Figure 4 A schematic diagram of a tumor located inside the liver provided in an embodiment of the present application;

[0068] Figure 5 A schematic diagram of segmenting a model projection of a three-dimensional tumor model provided in an embodiment of the present application;

[0069] Figure 6 A schematic diagram of a process for determining two-dimensional coordinates of vertices of a lesion surface mesh representing a minimum boundary point of the lesion provided in an embodiment of the present application;

[0070] Figure 7 A schematic diagram of a process for determining the minimum lesion cross section provided in an embodiment of the present application;

[0071] Figure 8 A schematic diagram of the minimum boundary point of a tumor marked according to an embodiment of the present application;

[0072] Fig. 9 For this application, a schematic diagram of a tumor safety boundary point for determining a tumor is provided in a mobile phone;

[0073] Fig.10 A schematic diagram of a safety boundary point provided in an embodiment of the present application;

[0074] Fig.11 A schematic diagram of a safe boundary point of a lesion on the surface of a three-dimensional model of internal organs provided in an embodiment of the present application;

[0075] Fig.12 A schematic diagram of a complete process of a method for generating a lesion boundary on the surface of a visceral three-dimensional model provided in an embodiment of the present application;

[0076] Fig.13 A complete flowchart of another method for generating lesion boundaries on the surface of a three-dimensional model of internal organs provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Among them, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0078] In the description of the embodiments of the present application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0079] A method for generating lesion boundaries on the surface of a three-dimensional model of internal organs and an electronic device provided in an embodiment of the present application can be applied to laparoscopic surgery. Currently, laparoscopic surgery usually uses laparoscopic augmented reality surgical navigation technology. The following first explains augmented reality (AR) technology and laparoscopic augmented reality surgical navigation technology.

[0080] AR technology is a new technology that "seamlessly" integrates real-world information and virtual-world information. It uses computers and other scientific technologies to simulate and superimpose physical information (visual information, sound, taste, touch, etc.) that is difficult to experience in a certain time and space range in the real world. It applies virtual information to the real world and is perceived by human senses, thereby achieving a sensory experience beyond reality. The real environment and virtual objects are superimposed on the same screen or space in real time and exist simultaneously. Augmented reality technology includes new technologies and methods such as multimedia, three-dimensional modeling, real-time video display and control, multi-sensor integration, real-time tracking and registration, and scene fusion.

[0081] Laparoscopic augmented reality surgical navigation technology is the main form of laparoscopic augmented reality navigation at present. It is mainly implemented by reconstructing the preoperative visceral three-dimensional model from CT / MRI medical images, and then registering the preoperative visceral three-dimensional model with the intraoperative laparoscopic image through image registration technology to generate the intraoperative visceral three-dimensional model. Finally, the augmented reality display technology is used to enhance the display of the intraoperative scene and realize image navigation.

[0082] In the related art, when laparoscopic surgery is performed, if the tumor is located inside the liver, it is necessary to manually mark the boundaries of the tumor on the surface of the three-dimensional liver model. The manual method of marking the boundaries of the tumor on the surface of the three-dimensional liver model is complicated and inefficient.

[0083] Based on the above problems, the present application provides a method for generating lesion boundaries on the surface of a 3D model of internal organs. The following describes the application scenario of the method for generating lesion boundaries on the surface of a 3D model of internal organs provided by the present application in conjunction with the accompanying drawings. Figure 1 As shown, it is a structural diagram of a laparoscopic augmented reality surgical navigation system. The laparoscopic augmented display surgical navigation system includes a laparoscope 2 and a laparoscopic augmented reality surgical navigation device 3, wherein the laparoscopic augmented reality surgical navigation device 3 includes an electronic device 31, and the electronic device 31 includes a processor 311 and a display screen 312:

[0084] The processor 311 is used to convert the three-dimensional coordinates of each lesion surface mesh vertex of the lesion into the two-dimensional coordinates of the lesion surface mesh vertex, wherein the three-dimensional coordinates of each lesion surface mesh vertex are obtained based on the registered lesion three-dimensional model; select the two-dimensional coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion from the two-dimensional coordinates of the lesion surface mesh vertex; determine the minimum lesion cross section of the lesion based on the selected two-dimensional coordinates of the lesion surface mesh vertex and the lesion center point, and calculate the minimum boundary point of the lesion on the surface of the lesion three-dimensional model according to the minimum lesion cross section; after obtaining the safety boundary point of the lesion based on the minimum boundary point and the preset distance on the surface of the lesion three-dimensional model, extend each safety boundary point of the lesion in the direction of the corresponding normal vector, and obtain the first intersection point with the surface of the visceral three-dimensional model as the safety boundary point of the lesion on the surface of the visceral three-dimensional model;

[0085] The display screen 32 is used to display the safety boundary points of the lesion on the surface of the three-dimensional model of internal organs on the laparoscopic image of the current view.

[0086] Figure 1 In the figure, 1 represents an internal organ. A preoperative three-dimensional model of the internal organ 1 is obtained before the operation. During the operation on the internal organ 1, the laparoscope 2 collects relevant information of the internal organ 1 in real time. The preoperative three-dimensional model of the internal organ 1 and the relevant information collected during the operation are aligned to obtain the aligned three-dimensional model, which is then superimposed on the laparoscopic image of the current view through enhanced display technology.

[0087] The method provided in the embodiments of the present application is not limited to Figure 1 The application scenario shown can also be used in other possible application scenarios, which are not limited in the embodiments of the present application.

[0088] like Figure 2 As shown, a flow chart of a method for generating a lesion boundary on the surface of a visceral three-dimensional model provided in an embodiment of the present application, the specific steps are as follows:

[0089] S201, converting the three-dimensional coordinates of each lesion surface mesh vertex into two-dimensional coordinates of the lesion surface mesh vertex, wherein the three-dimensional coordinates of each lesion surface mesh vertex are acquired based on the registered three-dimensional lesion model;

[0090] S202, selecting the two-dimensional coordinates of the lesion surface mesh vertices representing the minimum boundary point of the lesion from the two-dimensional coordinates of the lesion surface mesh vertices;

[0091] S203, determining the minimum lesion cross section of the lesion based on the two-dimensional coordinates of the selected lesion surface mesh vertices and the lesion center point, and calculating the minimum boundary point of the lesion on the surface of the lesion three-dimensional model according to the minimum lesion cross section;

[0092] S204. After obtaining the safety boundary points of the lesion based on the minimum boundary point and the preset distance on the surface of the three-dimensional model of the lesion, extend each safety boundary point of the lesion in the direction of the corresponding normal vector, and the obtained first intersection point with the surface of the three-dimensional model of the viscera is used as the safety boundary point of the lesion on the surface of the three-dimensional model of the viscera, and is displayed on the laparoscopic image of the current view.

[0093] The method for generating the boundary of a lesion on the surface of a 3D model of an internal organ provided in an embodiment of the present application first converts the 3D coordinates of each lesion surface mesh vertex in the registered 3D model of the lesion into the 2D coordinates of the lesion surface mesh vertex, then selects the 2D coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion from the 2D coordinates of the lesion surface mesh vertex, determines the minimum lesion cross section of the lesion based on the selected 2D coordinates of the lesion surface mesh vertex and the lesion center point, calculates the minimum boundary point of the lesion on the surface of the 3D model of the lesion based on the minimum lesion cross section, and finally obtains the safety boundary point of the lesion based on the minimum boundary point and a preset distance on the surface of the 3D model of the lesion, extends the safety boundary point of each lesion in the direction of the corresponding normal vector, and obtains the first intersection point with the surface of the 3D model of the internal organ as the safety boundary point of the lesion on the surface of the 3D model of the internal organ, and displays it on the laparoscopic image of the current view, thereby realizing the automatic generation of the safety boundary point of the lesion on the surface of the 3D model of the internal organ. Compared with the method of manually marking the boundary of the tumor on the surface of the 3D model of the internal organ, the method is simple to operate and has high efficiency.

[0094] In the embodiment of the present application, a visualization toolkit (Visualization toolkit, Vtk) can be used for three-dimensional modeling, and the constructed model will be displayed in the system of the modeling tool. The three-dimensional model of the internal organs and the three-dimensional model of the lesion in the present application are both registered models, and the lesion is located inside the internal organs. The internal organs in the present application can be the liver, and the lesion can be a tumor.

[0095] like Figure 3 As shown in the figure, the liver 3D model is obtained after 3D modeling using Vtk. Figure 3 It can be seen that the three-dimensional liver model is composed of multiple vertices and multiple triangular facets, and the multiple vertices and multiple triangular facets constitute the surface of the three-dimensional liver model.

[0096] In the embodiment of the present application, the three-dimensional model of the lesion, for example, the three-dimensional model of the tumor located in the liver, can refer to Figure 3 In the liver three-dimensional model, the lesion three-dimensional model is also composed of multiple vertices and multiple triangular faces, and multiple vertices and multiple triangular faces constitute the surface of the tumor three-dimensional model.

[0097] like Figure 4 Shown is a schematic diagram of a tumor located inside the liver provided in an embodiment of the present application.

[0098] When laparoscopic surgery is used, if the tumor is located inside the liver, it is necessary to mark the safe boundary points of the tumor on the surface of the liver 3D model. The embodiment of the present application provides a method for generating lesion boundaries on the surface of a visceral 3D model, which can automatically mark the safe boundary points of a tumor located inside the liver on the surface of the liver 3D model, with simple operation and high efficiency.

[0099] The embodiments of the present application are described in detail below.

[0100] In a specific implementation, firstly, the three-dimensional coordinates of the lesion surface mesh vertices of the registered three-dimensional lesion model are obtained, and then the three-dimensional coordinates of each lesion surface mesh vertex are converted into two-dimensional coordinates of the lesion surface mesh vertex.

[0101] Specifically, the lesion 3D model can be represented by vtkActor in the software. Use vtk's GetMapper and GetInput methods to extract the polydata information of vtkActor, where the polydata information includes the vertex information, triangle patch information, edge information, etc. of vtkActor, and then use GetPoints and GetCells methods to extract the 3D coordinates of each vertex in the lesion 3D model and the vertex information of each patch from the polydata information. The 3D coordinates of each vertex are the 3D coordinates of the vertices of the lesion surface mesh.

[0102] The three-dimensional coordinates of the lesion surface mesh vertices are converted into two-dimensional coordinates of the lesion surface mesh vertices. The SetCoordinateSystemToWorld, SetValue, and GetComputedDisplayValue methods of vtkCoordinate can be used to convert the three-dimensional coordinates of the lesion surface mesh vertices to obtain the two-dimensional coordinates of the lesion surface mesh vertices.

[0103] After obtaining the two-dimensional coordinates of the lesion surface mesh vertices, the two-dimensional coordinates of the lesion surface mesh vertices representing the minimum boundary point of the lesion are selected from the lesion surface mesh vertex coordinates.

[0104] Specifically, the model projection of the lesion three-dimensional model is segmented into a preset number of segmentation areas along the vertical direction, and the two-dimensional coordinates of the lesion surface mesh vertices representing the minimum boundary point of the lesion are selected according to the two-dimensional coordinates of the lesion surface mesh vertices in the segmentation areas.

[0105] First, the total length W of the model projection of the lesion three-dimensional model is confirmed, and then the segment length is determined according to the number of vertices to be extracted.

[0106] For example, if the total length W of the model projection is 15 cm and the number of vertices to be extracted is 15, the segment length is 3 cm.

[0107] like Figure 5 FIG. 1 is a schematic diagram of segmenting a model projection of a tumor three-dimensional model provided in an embodiment of the present application. Figure 5 The model projection is divided into 7 segments.

[0108] In one embodiment, when selecting the two-dimensional coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion, the two-dimensional coordinate with the largest horizontal coordinate, the two-dimensional coordinate with the smallest horizontal coordinate, the two-dimensional coordinate with the largest vertical coordinate, and the two-dimensional coordinate with the smallest vertical coordinate are determined among the two-dimensional coordinates of the lesion surface mesh vertex in each segmented area, and the determined two-dimensional coordinate with the largest horizontal coordinate, the two-dimensional coordinate with the smallest horizontal coordinate, the two-dimensional coordinate with the largest vertical coordinate, and the two-dimensional coordinate with the smallest vertical coordinate are used as the two-dimensional coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion.

[0109] like Figure 6 As shown, it is a schematic diagram of a process for determining the two-dimensional coordinates of the vertices of the lesion surface mesh representing the minimum boundary point of the lesion provided in an embodiment of the present application.

[0110] S601, confirming the total length of the model projection of the lesion three-dimensional model;

[0111] S602, determining the segment length according to the number of vertices to be extracted;

[0112] S603, segmenting the model projection of the lesion three-dimensional model according to the determined segment length;

[0113] S604, determining the two-dimensional coordinate with the largest abscissa, the two-dimensional coordinate with the smallest abscissa, the two-dimensional coordinate with the largest ordinate, and the two-dimensional coordinate with the smallest ordinate among the two-dimensional coordinates of the lesion surface mesh vertices in each segmented region;

[0114] S605 , using the determined two-dimensional coordinate with the largest abscissa, the two-dimensional coordinate with the smallest abscissa, the two-dimensional coordinate with the largest ordinate, and the two-dimensional coordinate with the smallest ordinate as the two-dimensional coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion.

[0115] In one embodiment, after selecting the two-dimensional coordinates of the lesion surface mesh vertices representing the minimum boundary point of the lesion, the minimum lesion cross-section of the lesion is determined based on the two-dimensional coordinates of the selected lesion surface mesh vertices and the lesion center point. Specifically, the first distance between any two selected two-dimensional coordinates of each lesion surface mesh vertices is calculated to obtain a group of two-dimensional coordinates of lesion surface mesh vertices with the largest first distance, and the line connecting the two-dimensional coordinates of the group of lesion surface mesh vertices with the largest distance is used as the central axis, and the second distance between the two-dimensional coordinates of each lesion surface mesh vertex and the central axis is calculated to determine the two-dimensional coordinates of two target lesion surface mesh vertices with the largest second distance on both sides of the central axis, and the two-dimensional coordinates of the target lesion surface mesh vertex are converted into three-dimensional coordinates, and the interface determined according to the three-dimensional coordinates of the two target lesion surface mesh vertices and the three-dimensional coordinates of the lesion center point is used as the minimum lesion cross-section of the lesion.

[0116] In a specific implementation, the getCenter method may be used to obtain the three-dimensional coordinates of the center point of the lesion.

[0117] For example, there are five two-dimensional coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion, namely A, B, C, D, and E. The first distance d11 between A and B, the first distance d12 between A and C, the first distance d13 between A and D, and the first distance d14 between A and E are calculated; the first distance d21 between B and C, the first distance d22 between B and D, and the first distance d23 between B and E are calculated; the first distance d31 between C and D, and the first distance d32 between C and E are calculated; and the first distance d41 between D and E is calculated. The first distance with the largest distance is selected from the first distance d11, the first distance d12, the first distance d13, the first distance d14, the first distance d21, the first distance d22, the first distance d23, the first distance d31, the first distance d32, and the first distance d41, such as d11, and the line connecting the two-dimensional coordinate A of the lesion surface mesh vertex and the two-dimensional coordinate B of the lesion surface mesh vertex is used as the central axis l.

[0118] After obtaining the central axis l, first search for the two-dimensional coordinates of the lesion surface mesh vertices on both sides of the central axis l, for example, A and C are located on one side of the central axis l, and B, D and E are located on the other side of the central axis l, and then calculate the vertical distance d1 from the two-dimensional coordinate A of the lesion surface mesh vertex to the central axis l, calculate the vertical distance d2 from the two-dimensional coordinate B of the lesion surface mesh vertex to the central axis l, calculate the vertical distance d3 from the two-dimensional coordinate C of the lesion surface mesh vertex to the central axis l, calculate the vertical distance d4 from the two-dimensional coordinate D of the lesion surface mesh vertex to the central axis l, and calculate the vertical distance d5 from the two-dimensional coordinate E of the lesion surface mesh vertex to the central axis l. If d1>d3, d2>d4>d5, then the two-dimensional coordinate A of the lesion surface mesh vertex and the two-dimensional coordinate B of the lesion surface mesh vertex are used as the two-dimensional coordinates of the target lesion surface mesh vertex.

[0119] After obtaining the two-dimensional coordinates of the two target lesion surface mesh vertices, they are converted into three-dimensional coordinates.

[0120] In one embodiment, the difference between any two three-dimensional coordinates of the three-dimensional coordinates of the two target lesion surface mesh vertices and the three-dimensional coordinates of the lesion center point can be calculated to obtain the two three-dimensional coordinate differences, and the normal vector is calculated based on the two three-dimensional coordinate differences. After the normal vector is obtained, vtkImageplane is used to obtain the minimum lesion cross-section based on any three-dimensional coordinate and the obtained normal vector.

[0121] For example, the three-dimensional coordinates of the two target lesion surface mesh vertices are A (x0, y0, z0) and B (x1, y1, z1), the three-dimensional coordinates of the lesion center are C (x2, y2, z2), vectors AB and AC are (x1-x0, y1-y0, z1-z0) and (x2-x0, y2-y0, z2-z0), and the normal vector is (v1, v2, v3). The normal vector of the plane must be perpendicular to all vectors in the plane. If two vectors are perpendicular, their dot product is 0. That is, it satisfies:

[0122] (x1-x0)*v1+(y1-y0)*v2+(z1-z0)*v3=0;

[0123] (x2-x0)*v1+(y2-y0)*v2+(z2-z0)*v3=0.

[0124] Solving the equations and eliminating the common parameters, we can obtain the plane normal vector as {(y1-y0)*(z2-z0)-(y2-y0)*(z1-z0), (z1-z0)*(x2-x0)-(z2-z0)*(x1-x0), (x1-x0)*(y2-y0)-(x2-x0)*(y1-y0)}.

[0125] After obtaining the plane normal vector, the minimum lesion cross section can be obtained according to any three-dimensional coordinate and the obtained plane normal vector.

[0126] like Figure 7 FIG. 1 is a flow chart of determining the minimum lesion cross section provided in an embodiment of the present application, which specifically includes the following steps:

[0127] S701, calculating the distance between the two-dimensional coordinates of the lesion surface mesh vertices of any two minimum boundary points representing the lesion, and obtaining the two-dimensional coordinates of the two lesion surface mesh vertices with the largest distance;

[0128] S702, taking the connecting line of the two-dimensional coordinates of the two lesion surface mesh vertices with the largest distance as the central axis, and calculating the vertical distance from the two-dimensional coordinates of each lesion surface mesh vertex representing the minimum boundary point of the lesion to the central axis;

[0129] S703, determining the two-dimensional coordinates of the two target lesion surface mesh vertices with the largest distance from the central axis on both sides of the central axis respectively;

[0130] S704, converting the two-dimensional coordinates of the two target lesion surface mesh vertices into three-dimensional coordinates;

[0131] S705 , obtaining the three-dimensional coordinates of the lesion center point of the lesion, and calculating the minimum lesion cross section of the lesion according to the three-dimensional coordinates of the two target lesion surface mesh vertices and the three-dimensional coordinates of the lesion center point.

[0132] After obtaining the minimum lesion cross section, the intersection of the minimum lesion cross section and the surface of the lesion three-dimensional model is calculated. The point corresponding to the obtained three-dimensional coordinates is the minimum boundary point of the lesion, that is, the minimum boundary point of the lesion on the surface of the lesion three-dimensional model.

[0133] Specifically, vtkCutter can be used to obtain the intersection of the minimum lesion cross section and the lesion three-dimensional model.

[0134] like Figure 8 The figure shows a schematic diagram of the minimum boundary point of a tumor marked according to an embodiment of the present application.

[0135] In one embodiment, after obtaining the minimum boundary point of the lesion on the surface of the lesion three-dimensional model, for each minimum boundary point, a preset distance is extended along the direction of the line connecting the lesion center point and the minimum boundary point of the lesion to obtain a safe boundary point of the lesion.

[0136] Specifically, the getCenter method can be used to obtain the center point of the lesion and convert it into a two-dimensional coordinate. According to the set extension distance, each point in the minimum boundary point is extended by a corresponding distance in the direction of the line connecting it and the center point of the lesion, such as 5mm, as the safe boundary point of the lesion.

[0137] For example, Fig. 9 As shown in the figure, the tumor is located inside the liver. There are five minimum boundary points of the tumor, namely point a, point b, point c, point d and point e, and the center point of the tumor is O. Extend point a in the direction of the line connecting point O and point a for a certain distance to obtain point a'; extend point b in the direction of the line connecting point O and point b for a certain distance to obtain point b'; extend point c in the direction of the line connecting point O and point c for a certain distance to obtain point c'; extend point d in the direction of the line connecting point O and point d for a certain distance to obtain point d'; extend point e in the direction of the line connecting point O and point e for a certain distance to obtain point e'. The obtained points a', b', c', d' and e' are the safe boundary points of the tumor.

[0138] like Fig.10 The figure shows a schematic diagram of a safety boundary point provided in an embodiment of the present application. Fig.10 In the figure, the white-marked points are safety boundary points.

[0139] In one embodiment, after obtaining the safety boundary points of the lesion, each safety boundary point of the lesion can be extended in the direction of the corresponding normal vector to obtain the intersection with the three-dimensional model of the internal organs. The obtained intersection is the safety boundary point of the lesion on the surface of the three-dimensional model of the internal organs.

[0140] like Fig.11 The figure is a schematic diagram of the safe boundary points of a lesion on the surface of a three-dimensional model of internal organs provided in an embodiment of the present application. Fig.11 In the figure, the black marked point at the other end of the dotted line is the safe boundary point of the lesion on the surface of the visceral three-dimensional model.

[0141] In one embodiment, after the minimum boundary point of the lesion is obtained, each minimum boundary point of the lesion is extended in the direction of the corresponding normal vector to obtain the intersection with the three-dimensional model of the internal organs. The obtained intersection is the minimum boundary point of the lesion on the surface of the three-dimensional model of the internal organs.

[0142] It should be noted that the obtained boundary points are all vertex coordinates of triangular patches on the viscera three-dimensional model.

[0143] Specifically, you can use the getOutput method of vtkTransformPolyDataFilter to obtain the coordinate information of the minimum boundary point of the lesion and the safety boundary point of the lesion, and then call the IntersectWithLine method of vtkOBBTree to obtain the intersection of the ray of the minimum boundary point in the direction of the normal vector and the visceral three-dimensional model, as well as the intersection of the ray of the safety boundary point in the direction of the normal vector and the visceral three-dimensional model.

[0144] In an embodiment of the present application, each boundary point is extended in the direction of the normal vector, and the intersection of the ray and the three-dimensional model of the internal organs is calculated. Finally, all the intersection sets are connected in sequence, which are the projections of the minimum boundary point and the safety boundary point on the surface of the three-dimensional model of the internal organs.

[0145] In a specific implementation, the coordinates corresponding to the minimum boundary point of the lesion on the surface of the three-dimensional model of the internal organs can be converted into two-dimensional coordinates, and after drawing based on the obtained two-dimensional coordinates, they can be displayed on the display interface.

[0146] In addition to displaying the minimum boundary points of the lesion on the surface of the three-dimensional model of the internal organs on the display interface, the safety boundary points of the lesion on the surface of the three-dimensional model of the internal organs can also be displayed on the display interface. Specifically, the coordinates corresponding to the safety boundary points of the lesion on the surface of the three-dimensional model of the internal organs are converted into two-dimensional coordinates, drawn based on the obtained two-dimensional coordinates, and displayed on the display interface.

[0147] Specifically, the minimum boundary points of the two-dimensional lesions on the surface of the three-dimensional model of internal organs and the safety boundary points of the two-dimensional lesions on the surface of the three-dimensional model of internal organs can be drawn using painter, QpainterPath and opengl related methods. After moving, rotating and scaling the three-dimensional model, the boundary of the lesion can still be calculated according to the same method and updated for display.

[0148] like Fig.12 FIG. 1 is a schematic diagram of a complete process of generating a lesion boundary on the surface of a visceral three-dimensional model provided in an embodiment of the present application. Specifically, the process includes the following steps:

[0149] S1201, obtaining the three-dimensional coordinates of the lesion surface mesh vertices of the registered three-dimensional lesion model;

[0150] S1202, converting the three-dimensional coordinates of each lesion surface mesh vertex into the two-dimensional coordinates of the lesion surface mesh vertex;

[0151] S1203, dividing the model projection of the lesion three-dimensional model into a preset number of segmentation areas along the vertical direction;

[0152] S1204, from the two-dimensional coordinates of the lesion surface mesh vertices in each segmented area, select the two-dimensional coordinate with the smallest horizontal coordinate as the left boundary point of the lesion, select the two-dimensional coordinate with the largest horizontal coordinate as the right boundary point of the lesion, select the two-dimensional coordinate with the smallest vertical coordinate as the lower boundary point of the lesion, and select the two-dimensional coordinate with the largest vertical coordinate as the upper boundary point of the lesion;

[0153] S1205, using the determined two-dimensional coordinates of the lesion surface as the two-dimensional coordinates of the lesion surface mesh vertices representing the minimum boundary point of the lesion;

[0154] S1206, calculating the distance between the two-dimensional coordinates of any two lesion surface mesh vertices representing the minimum boundary points of the lesion, and obtaining the two-dimensional coordinates of the two lesion surface mesh vertices with the largest distance;

[0155] S1207, taking the connecting line of the two-dimensional coordinates of the two lesion surface mesh vertices with the largest distance as the central axis, and calculating the vertical distance from the two-dimensional coordinates of each lesion surface mesh vertex representing the minimum boundary point of the lesion to the central axis;

[0156] S1208, determining the two-dimensional coordinates of the two target lesion surface mesh vertices with the largest distance from the central axis on both sides of the central axis respectively;

[0157] S1209, converting the two-dimensional coordinates of the two target lesion surface mesh vertices into three-dimensional coordinates;

[0158] S1210, obtaining the three-dimensional coordinates of the lesion center point of the lesion, and calculating the minimum lesion cross section of the lesion according to the three-dimensional coordinates of the two target lesion surface mesh vertices and the three-dimensional coordinates of the lesion center point;

[0159] S1211, calculating the intersection of the minimum lesion cross section and the surface of the lesion three-dimensional model, and the obtained three-dimensional coordinates are the minimum boundary point of the lesion;

[0160] The minimum boundary point of the lesion here is the minimum boundary point of the lesion on the surface of the lesion three-dimensional model.

[0161] S1212, converting the three-dimensional coordinates of the minimum boundary point of the lesion into two-dimensional coordinates;

[0162] S1213, for each minimum boundary point, extending a preset distance along the direction of the line connecting the center point of the lesion and the minimum boundary point of the lesion to obtain a safe boundary point of the lesion;

[0163] S1214, extending each safety boundary point of the lesion in the direction of the corresponding normal vector to obtain an intersection point with the visceral three-dimensional model, where the obtained intersection point is the safety boundary point of the lesion on the surface of the visceral three-dimensional model;

[0164] S1215, converting the obtained coordinates corresponding to the safety boundary points of the lesion on the surface of the visceral three-dimensional model into two-dimensional coordinates;

[0165] S1216. Draw the obtained two-dimensional coordinates of the safety boundary points of the lesion on the surface of the three-dimensional model of the internal organs and display them on the display interface.

[0166] like Fig.13 FIG. 1 is a schematic diagram of a complete process of generating a lesion boundary on the surface of another internal organ three-dimensional model provided in an embodiment of the present application. Specifically, the process includes the following steps:

[0167] S1301, obtaining the three-dimensional coordinates of the lesion surface mesh vertices of the registered three-dimensional lesion model;

[0168] S1302, converting the three-dimensional coordinates of each lesion surface mesh vertex into the two-dimensional coordinates of the lesion surface mesh vertex;

[0169] S1303, dividing the model projection of the lesion three-dimensional model into a preset number of segmentation areas along the vertical direction;

[0170] S1304, from the two-dimensional coordinates of the lesion surface mesh vertices in each segmented area, select the two-dimensional coordinate with the smallest horizontal coordinate as the left boundary point of the lesion, select the two-dimensional coordinate with the largest horizontal coordinate as the right boundary point of the lesion, select the two-dimensional coordinate with the smallest vertical coordinate as the lower boundary point of the lesion, and select the two-dimensional coordinate with the largest vertical coordinate as the upper boundary point of the lesion;

[0171] S1305, using the determined two-dimensional coordinates of the lesion surface as the two-dimensional coordinates of the lesion surface mesh vertices representing the minimum boundary point of the lesion;

[0172] S1306, calculating the distance between the two-dimensional coordinates of any two lesion surface mesh vertices representing the minimum boundary points of the lesion, and obtaining the two-dimensional coordinates of the two lesion surface mesh vertices with the largest distance;

[0173] S1307, taking the connecting line of the two-dimensional coordinates of the two lesion surface mesh vertices with the largest distance as the central axis, and calculating the vertical distance from the two-dimensional coordinates of each lesion surface mesh vertex representing the minimum boundary point of the lesion to the central axis;

[0174] S1308, determining the two-dimensional coordinates of the two target lesion surface mesh vertices with the largest distance from the central axis on both sides of the central axis respectively;

[0175] S1309, converting the two-dimensional coordinates of the two target lesion surface mesh vertices into three-dimensional coordinates;

[0176] S1310, obtaining the three-dimensional coordinates of the lesion center point of the lesion, and calculating the minimum lesion cross section of the lesion according to the three-dimensional coordinates of the two target lesion surface mesh vertices and the three-dimensional coordinates of the lesion center point;

[0177] S1311, calculating the intersection of the minimum lesion cross section and the surface of the lesion three-dimensional model, and the obtained three-dimensional coordinates are the minimum boundary point of the lesion;

[0178] The minimum boundary point of the lesion here is the minimum boundary point of the lesion on the surface of the lesion three-dimensional model.

[0179] S1312, extending each minimum boundary point of the lesion in the direction of the corresponding normal vector to obtain an intersection point with the visceral three-dimensional model, where the obtained intersection point is the minimum boundary point of the lesion on the surface of the visceral three-dimensional model;

[0180] S1313, converting the obtained coordinates corresponding to the minimum boundary points of the lesion on the surface of the visceral three-dimensional model into two-dimensional coordinates;

[0181] S1314. Draw the obtained two-dimensional coordinates of the minimum boundary point of the lesion on the surface of the three-dimensional model of the internal organs and display it on the display interface.

[0182] Based on the same inventive concept, the principle of solving the problem by the electronic device provided in the embodiment of the present application is similar to the principle of solving the problem by the above method, and the repeated parts will not be repeated.

[0183] The electronic device provided in the embodiment of the present application is Figure 1 The electronic device 31 shown in FIG. 3 includes a processor 311 and a display screen 312;

[0184] In one embodiment, the processor 311 is specifically configured to:

[0185] Segmenting the model projection of the three-dimensional model of the lesion into a preset number of segmentation areas along the vertical direction;

[0186] According to the two-dimensional coordinates of the lesion surface mesh vertices within the segmented area, the two-dimensional coordinates of the lesion surface mesh vertices representing the minimum boundary point of the lesion are selected.

[0187] In one embodiment, the processor 311 is specifically configured to:

[0188] Determine, among the two-dimensional coordinates of the vertices on the surface of the lesion mesh within the segmented region, the two-dimensional coordinate with the largest abscissa, the two-dimensional coordinate with the smallest abscissa, the two-dimensional coordinate with the largest ordinate, and the two-dimensional coordinate with the smallest ordinate;

[0189] The determined two-dimensional coordinate with the largest abscissa, the two-dimensional coordinate with the smallest abscissa, the two-dimensional coordinate with the largest ordinate, and the two-dimensional coordinate with the smallest ordinate are used as the two-dimensional coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion.

[0190] In one embodiment, the processor 311 is specifically configured to:

[0191] Calculating a first distance between any two selected two-dimensional coordinates of each lesion surface mesh vertex to obtain a set of two-dimensional coordinates of a group of lesion surface mesh vertices with the largest first distance;

[0192] Taking the line connecting the two-dimensional coordinates of the group of lesion surface mesh vertices with the largest distance as the central axis, calculating the second distance between the two-dimensional coordinates of each lesion surface mesh vertex and the central axis, and determining the two-dimensional coordinates of the two target lesion surface mesh vertices with the largest second distance on both sides of the central axis;

[0193] Converting the two-dimensional coordinates of the vertices of the target lesion surface mesh into three-dimensional coordinates;

[0194] An interface determined according to the three-dimensional coordinates of the two target lesion surface mesh vertices and the three-dimensional coordinates of the lesion center point is used as the minimum lesion cross section of the lesion.

[0195] In one embodiment, the processor 311 is specifically configured to:

[0196] The intersection point between the minimum lesion cross section of the lesion and the surface of the lesion three-dimensional model is calculated, and the points corresponding to the obtained multiple three-dimensional coordinates are used as the minimum boundary points of the lesion on the surface of the lesion three-dimensional model.

[0197] In one embodiment, the processor 311 is specifically configured to:

[0198] For each minimum boundary point of the lesion on the surface of the three-dimensional model of the lesion: according to the two-dimensional coordinates of the minimum boundary point, in the direction from the lesion center point to the minimum boundary point, the minimum boundary point is extended by the preset distance to obtain a safe boundary point of the lesion.

[0199] In one embodiment, the processor 311 is specifically configured to:

[0200] For each safety boundary point of the lesion: according to the triangular facet to which the safety boundary point of the lesion belongs, the normal vector coordinates corresponding to the safety boundary point of the lesion are calculated; according to the three-dimensional coordinates of the safety boundary point of the lesion and the normal vector coordinates, the ray corresponding to the safety boundary point of the lesion is determined; according to the three-dimensional coordinates of the surface mesh vertices of the three-dimensional model of the internal organs and the ray, the first intersection point of the safety boundary point of the lesion and the surface of the three-dimensional model of the internal organs is obtained;

[0201] The first intersection point obtained is used as a safe boundary point of the lesion on the surface of the three-dimensional model of the internal organs.

[0202] In one embodiment, before displaying on the laparoscopic image of the current view, the processor 311 is further configured to:

[0203] Converting the safety boundary points of the lesion on the surface of the three-dimensional model of the internal organs into two-dimensional coordinates;

[0204] Plot the transformed 2D coordinates.

[0205] In one embodiment, after calculating the minimum boundary point of the lesion on the surface of the lesion three-dimensional model according to the minimum lesion cross section, the processor 311 is further configured to:

[0206] Extending the minimum boundary point of each lesion on the surface of the lesion three-dimensional model in the direction of the corresponding normal vector to obtain a second intersection point with the surface of the viscera three-dimensional model;

[0207] The obtained second intersection point is used as the minimum boundary point of the lesion on the surface of the three-dimensional model of the internal organs, and is displayed on the laparoscopic image of the current view.

[0208] The embodiment of the present application provides a method for generating a lesion boundary on the surface of a visceral three-dimensional model and an electronic device. The method comprises: firstly converting the three-dimensional coordinates of each lesion surface mesh vertex in the registered lesion three-dimensional model into two-dimensional coordinates of the lesion surface mesh vertex, then selecting the two-dimensional coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion from the two-dimensional coordinates of the lesion surface mesh vertex, determining the minimum lesion cross section of the lesion based on the selected two-dimensional coordinates of the lesion surface mesh vertex and the lesion center point, and calculating the minimum boundary point of the lesion on the surface of the lesion three-dimensional model according to the minimum lesion cross section, and finally obtaining the safety boundary point of the lesion based on the minimum boundary point and a preset distance on the surface of the lesion three-dimensional model, extending the safety boundary point of each lesion in the direction of the corresponding normal vector, and obtaining the first intersection point with the surface of the visceral three-dimensional model as the safety boundary point of the lesion on the surface of the visceral three-dimensional model, and displaying it on the laparoscopic image of the current view, thereby realizing automatic generation of the safety boundary point of the lesion on the surface of the visceral three-dimensional model. Compared with the method of manually marking the boundary of the tumor on the surface of the visceral three-dimensional model, the operation is simple and the efficiency is high.

[0209] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0210] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0211] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0212] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0213] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for generating a lesion boundary on the surface of a 3D model of an internal organ, characterized in that: The method includes: Converting the three-dimensional coordinates of each lesion surface mesh vertex into two-dimensional coordinates of the lesion surface mesh vertex, wherein the three-dimensional coordinates of each lesion surface mesh vertex are obtained based on the registered three-dimensional lesion model; Selecting the two-dimensional coordinates of the vertices on the lesion surface mesh representing the minimum boundary point of the lesion from the two-dimensional coordinates of the vertices on the lesion surface mesh; Determine the minimum lesion cross section of the lesion based on the two-dimensional coordinates of the selected lesion surface mesh vertices and the lesion center point, and calculate the minimum boundary point of the lesion on the surface of the lesion three-dimensional model according to the minimum lesion cross section; After obtaining the safety boundary points of the lesion based on the minimum boundary point and the preset distance on the surface of the three-dimensional model of the lesion, each safety boundary point of the lesion is extended in the direction of the corresponding normal vector, and the first intersection point with the surface of the three-dimensional model of the viscera is used as the safety boundary point of the lesion on the surface of the three-dimensional model of the viscera and is displayed on the laparoscopic image of the current view.

2. The method according to claim 1, characterized in that The step of selecting the two-dimensional coordinates of the vertices on the lesion surface mesh representing the minimum boundary point of the lesion from the two-dimensional coordinates of the vertices on the lesion surface mesh comprises: Segmenting the model projection of the three-dimensional model of the lesion into a preset number of segmentation areas along the vertical direction; According to the two-dimensional coordinates of the lesion surface mesh vertices within the segmented area, the two-dimensional coordinates of the lesion surface mesh vertices representing the minimum boundary point of the lesion are selected.

3. The method according to claim 2, characterized in that The step of selecting the two-dimensional coordinates of the vertices on the surface of the lesion mesh representing the minimum boundary point of the lesion according to the two-dimensional coordinates of the vertices on the surface of the lesion mesh within the segmented area comprises: Determine, among the two-dimensional coordinates of the vertices on the surface of the lesion mesh within the segmented region, the two-dimensional coordinate with the largest abscissa, the two-dimensional coordinate with the smallest abscissa, the two-dimensional coordinate with the largest ordinate, and the two-dimensional coordinate with the smallest ordinate; The determined two-dimensional coordinate with the largest abscissa, the two-dimensional coordinate with the smallest abscissa, the two-dimensional coordinate with the largest ordinate, and the two-dimensional coordinate with the smallest ordinate are used as the two-dimensional coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion.

4. The method according to claim 1, characterized in that The step of determining the minimum lesion cross section of the lesion based on the two-dimensional coordinates of the selected lesion surface mesh vertices and the lesion center point comprises: Calculating a first distance between any two selected two-dimensional coordinates of each lesion surface mesh vertex to obtain a set of two-dimensional coordinates of a group of lesion surface mesh vertices with the largest first distance; Taking the line connecting the two-dimensional coordinates of the group of lesion surface mesh vertices with the largest distance as the central axis, calculating the second distance between the two-dimensional coordinates of each lesion surface mesh vertex and the central axis, and determining the two-dimensional coordinates of the two target lesion surface mesh vertices with the largest second distance on both sides of the central axis; Converting the two-dimensional coordinates of the vertices of the target lesion surface mesh into three-dimensional coordinates; An interface determined according to the three-dimensional coordinates of the two target lesion surface mesh vertices and the three-dimensional coordinates of the lesion center point is used as the minimum lesion cross section of the lesion.

5. The method according to claim 4, characterized in that The step of calculating the minimum boundary point of the lesion on the surface of the three-dimensional model of the lesion according to the minimum lesion cross section includes: The intersection point between the minimum lesion cross section of the lesion and the surface of the lesion three-dimensional model is calculated, and the points corresponding to the obtained multiple three-dimensional coordinates are used as the minimum boundary points of the lesion on the surface of the lesion three-dimensional model.

6. The method according to claim 1, characterized in that The step of obtaining the safe boundary point of the lesion based on the minimum boundary point and the preset distance on the surface of the lesion three-dimensional model comprises: For each minimum boundary point of the lesion on the surface of the three-dimensional model of the lesion: according to the two-dimensional coordinates of the minimum boundary point, in the direction from the lesion center point to the minimum boundary point, the minimum boundary point is extended by the preset distance to obtain a safe boundary point of the lesion.

7. The method according to claim 1, characterized in that The step of extending the safety boundary point of each lesion upward along the corresponding normal vector to obtain a first intersection point with the surface of the three-dimensional model of the internal organs as the safety boundary point of the lesion on the surface of the three-dimensional model of the internal organs comprises: For each safety boundary point of the lesion: according to the triangular facet to which the safety boundary point of the lesion belongs, the normal vector coordinates corresponding to the safety boundary point of the lesion are calculated; according to the three-dimensional coordinates of the safety boundary point of the lesion and the normal vector coordinates, the ray corresponding to the safety boundary point of the lesion is determined; according to the three-dimensional coordinates of the surface mesh vertices of the three-dimensional model of the internal organs and the ray, the first intersection point of the safety boundary point of the lesion and the surface of the three-dimensional model of the internal organs is obtained; The first intersection point obtained is used as a safe boundary point of the lesion on the surface of the three-dimensional model of the internal organs.

8. The method according to claim 7, characterized in that Before displaying on the laparoscopic image of the current view, the method further includes: Converting the safety boundary points of the lesion on the surface of the three-dimensional model of the internal organs into two-dimensional coordinates; Plot the transformed 2D coordinates.

9. The method according to any one of claims 1 to 8, characterized in that: After calculating the minimum boundary point of the lesion on the surface of the lesion three-dimensional model according to the minimum lesion cross section, the method further includes: Extending the minimum boundary point of each lesion on the surface of the lesion three-dimensional model in the direction of the corresponding normal vector to obtain a second intersection point with the surface of the viscera three-dimensional model; The obtained second intersection point is used as the minimum boundary point of the lesion on the surface of the three-dimensional model of the internal organs, and is displayed on the laparoscopic image of the current view.

10. An electronic device, characterized in that: The electronic device comprises a processor and a display screen; The processor is used to convert the three-dimensional coordinates of each lesion surface mesh vertex of the lesion into the two-dimensional coordinates of the lesion surface mesh vertex, wherein the three-dimensional coordinates of each lesion surface mesh vertex are obtained based on the registered lesion three-dimensional model; select the two-dimensional coordinates of the lesion surface mesh vertex representing the minimum boundary point of the lesion from the two-dimensional coordinates of the lesion surface mesh vertices; determine the minimum lesion cross section of the lesion based on the selected two-dimensional coordinates of the lesion surface mesh vertex and the lesion center point, and calculate the minimum boundary point of the lesion on the surface of the lesion three-dimensional model according to the minimum lesion cross section; after obtaining the safety boundary point of the lesion based on the minimum boundary point and the preset distance on the surface of the lesion three-dimensional model, extend each safety boundary point of the lesion in the direction of the corresponding normal vector, and obtain the first intersection point with the surface of the visceral three-dimensional model as the safety boundary point of the lesion on the surface of the visceral three-dimensional model; The display screen is used to display the safe boundary points of the lesion on the surface of the three-dimensional model of the internal organs on the laparoscopic image of the current view.

Images