Curved surface planning method and device, computer equipment and surgical robot system

Abstract

The invention relates to a curved surface planning method and device, computer equipment and a surgical robot system. The method comprises the steps of obtaining a to-be-processed model corresponding to medical image data based on the medical image data; determining a reference point and a control point of the to-be-processed model, and generating a planar model for the to-be-processed model according to the reference point and the control point; the reference point is located in a target area in the to-be-processed model; the control points are in the surface area of the to-be-processed model; mapping the plane model to a to-be-processed model to obtain a curved surface planning result for the to-be-processed model; the curved surface planning result comprises a curved surface obtained by cutting the target area from the to-be-processed model. By adopting the method, the curved surface planning efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of medical technology, and in particular to a surface planning method, apparatus, computer equipment, surgical robot system, storage medium and computer program product. Background Art

[0002] With the development of minimally invasive technology, in order to achieve better surgical results, surgical planning is usually carried out on the lesion tissue to be removed before surgery, and a planning plan for the resection surface is formulated based on the three-dimensional positional relationship between the lesion and the normal tissue around the lesion.

[0003] In the existing planning technology, the operator needs to outline and modify the lines on the two-dimensional image frame by frame, and finally form a three-dimensional surface by interpolating multiple layers of two-dimensional curves. This solution is inefficient, the process is complicated, and a huge amount of operations are required to obtain the ideal planning surface. Summary of the invention

[0004] Based on this, it is necessary to provide a surface planning method, device, computer equipment, surgical robot system, computer-readable storage medium and computer program product that can improve the efficiency of surface planning in order to address the above-mentioned technical problems.

[0005] In a first aspect, the present application provides a surface planning method, the method comprising:

[0006] Based on the medical image data, obtaining a to-be-processed model corresponding to the medical image data;

[0007] Determine the reference point and control point of the model to be processed, and generate a plane model for the model to be processed according to the reference point and the control point; the reference point is in the target area of ​​the model to be processed; the control point is in the surface area of ​​the model to be processed;

[0008] The plane model is mapped onto the model to be processed to obtain a surface planning result for the model to be processed; the surface planning result includes a surface for cutting off the target area from the model to be processed.

[0009] In one embodiment, based on the medical image data, obtaining a to-be-processed model corresponding to the medical image data includes:

[0010] Based on the medical imaging data, the segmentation results for the area to be segmented and the surface area are obtained;

[0011] Based on the segmentation results and the preset safety boundary of the area to be segmented, a model to be processed including the target area is obtained.

[0012] In one embodiment, determining reference points and control points of a model to be processed, and generating a plane model for the model to be processed according to the reference points and the control points, includes:

[0013] Based on the target area, determine the reference point;

[0014] Based on the reference points and the surface area, control points are determined;

[0015] Processing reference points and control points to determine reference normal vectors for the plane model;

[0016] Generate a plane model based on a reference normal vector.

[0017] In one embodiment, determining a reference point based on the target area includes:

[0018] Determine the center of gravity of the target area or the center of gravity of the bounding box of the target area as a reference point;

[0019] Based on the reference points and surface areas, control points are determined, including:

[0020] The point on the surface area that is closest to the reference point is determined as the control point;

[0021] Process reference points and control points to determine the reference normal vector for the plane model, including:

[0022] The direction from the reference point to the control point is determined as the reference normal vector for the plane model.

[0023] In one embodiment, generating a plane model according to a reference normal vector includes:

[0024] According to the reference normal vector, the initial plane where the plane model is located is determined; the initial plane is a plane where the control point deviates from the model to be processed by a preset distance along the reference normal vector; based on the bounding box for the target area, the boundary parameters of the plane model are determined;

[0025] The initial plane is meshed and the plane model is generated by combining the boundary parameters of the plane model.

[0026] In one embodiment, determining the boundary parameters of the plane model based on the bounding box of the target area includes:

[0027] Based on the projection of the bounding box of the target area on the initial plane, the two-dimensional surface corresponding to the projection is enlarged to determine the boundary parameters of the plane model.

[0028] In one embodiment, mapping the plane model to the model to be processed to obtain a surface planning result for the model to be processed also includes:

[0029] Based on the reference normal vector, each grid point in the plane model is divided into a plurality of first grid points and a plurality of second grid points;

[0030] Mapping each first grid point to the target area of ​​the model to be processed to obtain a plurality of target mapping points; the target mapping point is the farthest mapping point of the first grid point in the target area based on the reference normal vector;

[0031] According to each target mapping point and each second grid point, the surface planning result is obtained by fitting.

[0032] In one of the embodiments, the first grid point is a grid point where the projection on the reference normal vector intersects with the target area; the second grid point is a grid point where the projection on the reference normal vector does not intersect with the target area.

[0033] In a second aspect, the present application provides a surface planning device, the device comprising:

[0034] A model-to-be-processed acquisition module is used to acquire the model-to-be-processed corresponding to the medical image data based on the medical image data;

[0035] A plane model generation module is used to determine the reference points and control points of the model to be processed, and generate a plane model for the model to be processed based on the reference points and the control points; the reference points are located in the target area of ​​the model to be processed; and the control points are located in the surface area of ​​the model to be processed;

[0036] The surface planning result acquisition module is used to map the plane model onto the model to be processed to obtain the surface planning result for the model to be processed; the surface planning result is used to indicate the path for cutting the target area from the model to be processed.

[0037] In a third aspect, the present application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0038] In a fourth aspect, the present application provides a surgical robot system, the system including a surgical robot; the surgical robot is used to acquire medical imaging data and acquire a model to be processed corresponding to the medical imaging data, and the surgical robot executes the above method.

[0039] In a fifth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the computer program is executed by a processor.

[0040] In a sixth aspect, the present application provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0041] The above-mentioned surface planning method, device, computer equipment, surgical robot system, storage medium and computer program product, the method can perform surface planning on the model to be processed by obtaining the model to be processed corresponding to the medical image data based on the medical image data, without outlining and modifying the lines on the two-dimensional image frame by frame; determine the reference points and control points of the model to be processed, and generate a plane model for the model to be processed based on the reference points and the control points, and the plane model can be used as a reference plane for generating the resection surface of the target area; wherein the reference point is within the target area in the model to be processed; the control point is within the surface area of ​​the model to be processed; the plane model is mapped onto the model to be processed to obtain the surface planning result for the model to be processed; the surface planning result includes the surface for resecting the target area from the model to be processed, without manually selecting and editing the points on the model to be processed multiple times, the interaction is simple, and a resection surface can be generated with very few operations, and the planned cutting surface has good effect, thereby improving the efficiency of surface planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of a flow chart of a surface planning method in one embodiment;

[0043] Figure 2 is a schematic diagram of a surface planning method in one embodiment;

[0044] Figure 3 A schematic diagram of a flow chart of a surface planning step in an embodiment;

[0045] Figure 4 is a schematic diagram of a surface planning method in another embodiment;

[0046] Figure 5 A schematic flow chart of a surface planning step in another embodiment;

[0047] Figure 6 A schematic diagram of a specific process of a surface planning step in an embodiment;

[0048] Figure 7 It is a flowchart of a surface planning step in another embodiment;

[0049] Figure 8 A schematic diagram of a flow chart of a surface planning step in yet another embodiment;

[0050] Fig. 9 A schematic diagram of a surface planning method in an example;

[0051] Fig.10 is a structural block diagram of a surface planning device in one embodiment;

[0052] Fig.11FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] Traditional surface planning schemes require manual outlining and modification of lines on two-dimensional images frame by frame, and finally a three-dimensional surface is formed by interpolation of multiple layers of two-dimensional curves. Three-dimensional surface planning technology can intuitively display the three-dimensional position of the resection surface, the location of the lesion, and the location of other tissues, and supports manual observation, analysis, and modification of the resection surface planning from any angle. This scheme requires strong film reading ability and relies on film reading experience. It is inefficient and the process is complicated. It requires a huge amount of operations to obtain an ideal planning surface.

[0055] Existing solutions have problems such as insufficient software interaction to provide effective convenience and a complicated process for generating surfaces. For example, it is necessary to manually formulate a plane, edit the plane, generate control points, and edit the control points, and finally generate a surface through the control points and the surface reconstruction algorithm. The interaction steps are cumbersome and complicated. For example, the surface editing process is not real-time. After the operation of editing the control points is completed, the adjusted surface is obtained according to the surface reconstruction algorithm. During the process, only the position of the control points can be obtained, and the result of real-time update of the surface cannot be obtained. The updated result can only be obtained through the surface reconstruction algorithm after the editing is completed. At the same time, since the editing method is to edit the control points, the editing range of the surface editing cannot be fed back on the surface display, and the range cannot be edited directly. As a result, it is necessary to understand the surface reconstruction principle during the editing process, and independently judge the editing range according to the distribution of control points, or it is necessary to obtain the desired surface model through multiple attempts and continuous exploration.

[0056] In summary, the current surface planning system does not have sufficient convenience and operability, and the overly cumbersome and complex interactive operations will cause the operator to spend a lot of time to develop a planning plan, which is inefficient. In order to solve the above problems, the present application provides a surface planning method, device, computer equipment, surgical robot system, storage medium and computer program product.

[0057] In one embodiment, a surface planning method is provided, such as Figure 1 As shown, the method includes:

[0058] Step 110, based on the medical image data, obtaining a to-be-processed model corresponding to the medical image data;

[0059] Specifically, medical imaging data can be obtained, and the medical imaging data can be medical imaging data for a target part, for example, at least one of X-ray projection imaging data, CT imaging data, ultrasound imaging data, magnetic resonance imaging data, and radionuclide imaging data. Based on the medical imaging data, a model to be processed corresponding to the medical imaging data can be generated, and the model to be processed can include a three-dimensional reconstruction model corresponding to the medical imaging data. The target part corresponding to the medical imaging data can include lesion tissue. By obtaining the model to be processed corresponding to the medical imaging data based on the medical imaging data, the model to be processed can be surface planned, without outlining and modifying lines on the two-dimensional image frame by frame, thereby improving the efficiency of surface planning.

[0060] In some examples, the medical image data may be pre-stored medical image data obtained from a memory, or may be medical image data downloaded from the Internet, etc. Figure 2 As shown in (a), the model to be processed can be generated based on medical imaging data of the target site, and the model to be processed includes a target area for characterizing the location range of the lesion tissue, and then the surface required to cut the target area from the model to be processed can be planned.

[0061] Step 120, determining reference points and control points of the model to be processed, and generating a plane model for the model to be processed based on the reference points and the control points; the reference points are located in the target area of ​​the model to be processed; and the control points are located in the surface area of ​​the model to be processed;

[0062] Specifically, the reference points and control points of the model to be processed can be determined, where Figure 2 As shown in (b), the reference point can be located in the target area of ​​the model to be processed, and is used to characterize the location of the target area. The reference point can be any point in the target area of ​​the model to be processed; the target area can be an area that needs to be removed from the model to be processed (for example, the target area can be used to characterize the location range of the lesion tissue); the control point can be located in the surface area of ​​the model to be processed, and the control point can be a point required for generating the surface planning result. For example, the position of the control point can be determined based on the position of the reference point. Figure 2As shown in (c), based on the reference points and control points, a plane model for the model to be processed can be generated. The plane model can be a model with a boundary on a plane outside the model to be processed, and can be at a certain distance from the surface area of ​​the model to be processed. Specifically, the plane model can be determined based on the straight line where the reference points and control points are located. For example, the plane model can be perpendicular to the straight line where the reference points and control points are located, or it can be at other angles. By determining the reference points and control points of the model to be processed, and generating a plane model for the model to be processed based on the reference points and control points, the plane model can be used as a reference plane for generating the resection surface of the target area, without the need to manually select and edit points on the model to be processed multiple times, thereby improving the efficiency of surface planning.

[0063] In some examples, the lesion tissue in the medical imaging data can be specified by the operator, so as to determine the target area in the model to be processed; the reference point and the control point can be specified by the operator or automatically generated. For example, the reference point can be the center point of the target area in the model to be processed, and the control point can be the point closest to the reference point in the surface area of ​​the model to be processed. The locations of the reference points and control points can all support editing. The interaction method is not fixed, and can support selecting the corresponding area on the two-dimensional / three-dimensional image with the mouse, or selecting the corresponding target from the organization list in the software interaction interface.

[0064] Step 130 , mapping the plane model onto the model to be processed to obtain a surface planning result for the model to be processed; the surface planning result includes a surface for cutting off the target area from the model to be processed.

[0065] Specifically, the plane model can include several points, such as Figure 2 As shown in (d) in FIG. 1 , the plane model can be mapped to the model to be processed. For example, the plane model can be mapped to the model to be processed along the straight line direction where the reference point and the control point are located. Then, the corresponding points obtained by mapping several points on the plane model can be determined on the model to be processed. Combined with the target area, for example, Figure 2 As shown in (e) in the figure, combined with the boundary information of the target area, the surface planning result for the model to be processed can be obtained. The cut surface included in the surface planning result can be used to cut the target area from the model to be processed. By mapping the plane model to the model to be processed, the surface planning result for the model to be processed is obtained. The interaction is simple and only a few operations are required to generate a cut surface. In addition, the planned cutting surface has a good effect, which improves the efficiency of the surface planning process.

[0066] In some examples, the target area can be used to characterize the location range including the lesion tissue, that is, the target area can be the area to be segmented (the area to be segmented can correspond to the area where the lesion tissue is located), or it can be the area including the area to be segmented (the target area can correspond to the area covering the lesion tissue), so that the generated resection surface has sufficient operating space to completely remove the area to be segmented.

[0067] The method of the embodiment of the present application can perform surface planning on the model to be processed by obtaining the model to be processed corresponding to the medical image data based on the medical image data, without outlining and modifying the lines on the two-dimensional image frame by frame; determine the reference points and control points of the model to be processed, and generate a plane model for the model to be processed based on the reference points and the control points, and the plane model can be used as a reference plane for generating the resection surface of the target area; wherein the reference point is within the target area in the model to be processed; the control point is within the surface area of ​​the model to be processed; the plane model is mapped onto the model to be processed to obtain the surface planning result for the model to be processed; the surface planning result includes the surface for resecting the target area from the model to be processed, and there is no need to manually select and edit the points on the model to be processed multiple times. The interaction is simple, and a resection surface can be generated with very few operations, and the planned cutting surface has good effect, thereby improving the efficiency of surface planning.

[0068] In one embodiment, if Figure 3 As shown, based on the medical image data, the model to be processed corresponding to the medical image data is obtained, including:

[0069] Step 310, obtaining segmentation results for the area to be segmented and the surface area based on the medical image data;

[0070] Step 320: Based on the segmentation result and the preset safety boundary of the area to be segmented, a model to be processed including the target area is obtained.

[0071] Specifically, the medical image data may be data of medical images of a target part, and based on the medical image data, the segmentation results of the area to be segmented and the surface area may be obtained, for example, Figure 4 As shown in (a), the segmentation results for the target part can be obtained, including the segmentation results for the lesion tissue and the other organ tissues. Based on the segmentation results for the lesion tissue and the other organ tissues, the segmentation results for the area to be segmented can be determined, that is, the area corresponding to the lesion tissue is taken as the area to be segmented; based on the tissue structure of the target part, the segmentation results for the surface area can be determined, for example, the area corresponding to the surface of the tissue or organ corresponding to the target part is taken as the surface area. Figure 4As shown in (b), a preset safety boundary can be set for the area to be segmented to determine the target area in the model to be processed, that is, the target area can include the area to be segmented and the preset safety boundary. The model to be processed determined in the above manner is expanded according to the preset safety boundary to obtain the target area, so that the position of the resection surface can be determined, and the generated resection surface has enough operating space to ensure that the area to be segmented is completely resected.

[0072] In some examples, the preset safety margin can be the required extension distance for completely removing the preset safety margin from the model to be processed while ensuring the safety of the removal. The size of the preset safety margin can be specified by the operator. For example, the preset safety margin for the area to be segmented can be 10 mm wide. The preset safety margin can be input by the operator in a text box, a slider, or other similar controls in the software interactive interface.

[0073] In one embodiment, if Figure 5 As shown, the reference points and control points of the model to be processed are determined, and a plane model for the model to be processed is generated according to the reference points and the control points, including:

[0074] Step 510, determining a reference point based on the target area;

[0075] Step 520, determining a control point based on the reference point and the surface area;

[0076] Step 530, processing the reference points and the control points to determine a reference normal vector for the plane model;

[0077] Step 540: Generate a plane model according to the reference normal vector.

[0078] Specifically, if Figure 4 As shown in (c) in FIG. 1 , a point inside the target area can be selected as a reference point. For example, the reference point can be the center of gravity of the target area, or a point within a certain distance range near the center of gravity. Without considering the density of the tissue corresponding to the target area, the reference point can be the centroid of the target area, or a point within a certain distance range near the centroid. Further, based on the reference point and the surface area, a control point can be determined on the surface area. For example, any point on the surface area that is close to the reference point can be determined as a control point. Figure 4As shown in (d) in the figure, the reference points and control points can be processed to determine the reference normal vector for the plane model. For example, the reference normal vector of the plane model can be determined based on the straight line where the reference points and the control points are located. The reference normal vector of the plane model can be parallel to the straight line where the reference points and the control points are located, or it can be at an angle within a certain range. The plane model can be generated based on the reference normal vector. The plane model can be perpendicular to the reference normal vector, or it can be at an angle within a certain range. The above method can automatically generate a plane model without specifying multiple control points. The resection surface can be automatically generated based on the obtained plane model, thereby improving the efficiency of surface planning.

[0079] In some examples, the reference point may be the center of gravity of the target area; the reference normal vector may be parallel to the straight line where the reference point and the control point are located, for example, the reference normal vector may be in the direction from the reference point to the control point, or may be in the direction from the control point to the reference point; the distance between the plane model and the control point may be determined based on the distance between the reference point and the control point, for example, the distance between the plane model and the control point may be one-fourth of the distance between the reference point and the control point, and based on the reference normal vector, the control point may be moved the above distance in the direction away from the model to be processed to obtain the center of the initial plane, thereby constructing the initial plane where the plane model is located.

[0080] In one embodiment, if Figure 6 As shown, based on the target area, reference points are determined, including:

[0081] Step 610, determining the centroid of the target area or the centroid of the bounding box of the target area as a reference point;

[0082] Specifically, the reference point can be the centroid of the target area. Without considering the density of the tissue corresponding to the target area, the reference point can be the centroid of the target area. The reference point can also be the centroid of the bounding box of the target area. Based on the target area, a bounding box of the target area can be generated, for example, a rectangular bounding box, and the centroid of the bounding box of the target area is determined as the reference point. By determining the reference point in the above manner, the target area can be accurately located, thereby improving the accuracy of surface planning.

[0083] In some examples, the bounding box of the target area may be an axis-aligned bounding box (AABB).

[0084] Based on the reference points and surface areas, control points are determined, including:

[0085] Step 620, determining the point on the surface area that is closest to the reference point as a control point;

[0086] Specifically, the point on the surface area that is closest to the reference point can be determined as a control point, thereby shortening the path of the resected surface obtained by the surface planning result. By determining the control point in the above manner, the path of the surface planning can be shortened and the quality of the surface planning can be improved.

[0087] Process reference points and control points to determine the reference normal vector for the plane model, including:

[0088] Step 630: determine the direction from the reference point to the control point as a reference normal vector for the plane model.

[0089] Specifically, the direction from the reference point to the control point can be determined as the reference normal vector for the plane model, and then based on the reference normal vector, the initial plane where the plane model is located can be determined in turn, and each grid point in the plane model can be mapped to the target area of ​​the model to be processed to generate the corresponding resection surface. Determining the reference normal vector in the above manner can achieve accurate mapping based on the reference normal vector, thereby improving the accuracy of surface planning.

[0090] In one embodiment, if Figure 7 As shown, a plane model is generated according to a reference normal vector, including:

[0091] Step 710, determining the initial plane where the plane model is located according to the reference normal vector; the initial plane is the plane where the control point deviates from the model to be processed by a preset distance along the reference normal vector;

[0092] Step 720, determining boundary parameters of the plane model based on the bounding box of the target area;

[0093] Step 730, meshing the initial plane, and generating a plane model in combination with the boundary parameters of the plane model.

[0094] Specifically, the initial plane where the plane model is located can be determined according to the reference normal vector. The initial plane can be perpendicular to the reference normal vector or at a certain range of angles with the reference normal vector. The initial plane is a plane where the control point deviates from the model to be processed by a preset distance along the reference normal vector; based on the bounding box for the target area, the boundary parameters of the plane model can be determined. The boundary parameters of the plane model can reflect the size of the plane model. The size of the plane model can be the projection size of the bounding box of the target area projected to the initial plane, or it can be larger than the projection size of the bounding box of the target area projected to the initial plane. For example, the projection of the bounding box of the target area projected to the initial plane is extended according to the preset boundary to determine the size of the plane model; the size of the plane model can also be directly determined according to the size of the bounding box for the target area. For example, the size of the plane model can be equal to the size of the maximum two-dimensional bounding box for the target area, or it can be larger than the size of the maximum two-dimensional bounding box for the target area. Further, the initial plane can be meshed, for example, the mesh within the size of the plane model is retained to generate a plane model. In the above manner, a plane model can be automatically generated as a reference for the subsequent generation of a resection surface, thereby improving the efficiency of obtaining the surface planning result for the model to be processed.

[0095] In some examples, the initial plane may be perpendicular to the reference normal vector; the preset distance may be one-fourth of the distance between the reference point and the control point; based on the reference normal vector, the control point may be moved away from the model to be processed by the preset distance to obtain the center of the initial plane, thereby constructing the initial plane where the plane model is located; the bounding box for the target area is projected onto the initial plane to obtain the projection of the maximum two-dimensional bounding box, for example, the projection is performed based on the reference normal vector; and to ensure the integrity of the planned approach cutting surface, the finite plane of the rectangular area (for example, 1.6i×1.6j) may be enlarged by a certain proportion based on the projection size of the maximum two-dimensional bounding box (for example, i×j), and the finite plane of the rectangular area is determined as the plane model. The plane model may be meshed, for example, the finite plane of the rectangular area may be divided into an m×n grid form, wherein m and n are both positive integers, and the values ​​of m and n may be the same (for example, m=n=75) or different.

[0096] In one embodiment, determining the boundary parameters of the plane model based on the bounding box of the target area includes:

[0097] Based on the projection of the bounding box of the target area on the initial plane, the two-dimensional surface corresponding to the projection is enlarged to determine the boundary parameters of the plane model.

[0098] Specifically, based on the projection of the bounding box of the target area on the initial plane, the two-dimensional surface corresponding to the projection can be enlarged to determine the boundary parameters of the plane model, so that the size of the plane determined based on the boundary parameters of the plane model can be larger than the size of the two-dimensional surface corresponding to the projection of the bounding box of the target area on the initial plane. Figure 4 As shown in (f) in the figure, the resection surface can be approximated as a cone, and the two-dimensional surface corresponding to the projection cannot accommodate the complete resection surface. Through the above method, it can be ensured that when each grid point of the plane model is mapped to the model to be processed, the target area in the model to be processed can be covered, and the complete resection surface can be accommodated on the plane determined by the boundary parameters of the plane model, thereby ensuring that the generated resection surface can completely resect the area to be segmented.

[0099] In one embodiment, if Figure 8 As shown, the plane model is mapped onto the model to be processed to obtain the surface planning result for the model to be processed, and also includes:

[0100] Step 810, dividing each grid point in the plane model into a plurality of first grid points and a plurality of second grid points based on the reference normal vector;

[0101] Step 820, mapping each first grid point to the target area of ​​the model to be processed to obtain a plurality of target mapping points; the target mapping point is the farthest mapping point of the first grid point in the target area based on the reference normal vector;

[0102] Step 830: fitting the surface planning result according to each target mapping point and each second grid point.

[0103] Specifically, if Figure 4 As shown in (e), each grid point in the plane model can be projected to the model to be processed based on the reference normal vector, for example, based on the reference normal vector, wherein a portion of the grid points can be projected into the target area of ​​the model to be processed, and another portion of the grid points cannot be projected into the target area of ​​the model to be processed, and then each grid point in the plane model can be divided into a plurality of first grid points and a plurality of second grid points, for example, the first grid points are grid points that can be projected into the target area of ​​the model to be processed based on the reference normal vector, and the second grid points are grid points that cannot be projected into the target area of ​​the model to be processed based on the reference normal vector. Each first grid point can be mapped into the target area of ​​the model to be processed, for example, based on the reference normal vector, each first grid point is mapped one by one into the target area of ​​the model to be processed, and a plurality of target mapping points are obtained, wherein the target mapping point is the farthest mapping point of the first grid point in the target area based on the reference normal vector; as shown in FIG. Figure 4As shown in (f), each second grid point does not need to be mapped. According to each target mapping point and each second grid point, the surface planning result is fitted, and the surface planning result includes the resection surface obtained by fitting each target mapping point and each second grid point. By dividing each grid point in the plane model into the first grid point and the second grid point for mapping respectively in the above manner, the integrity of the surface planning result obtained by planning can be ensured. The obtained resection surface includes the resection direction of the resection surface, which has better reference in practical applications.

[0104] In some examples, based on the reference normal vector, each first grid point is mapped to the farthest point in the target area of ​​the model to be processed through an editing algorithm to obtain a deformed surface model, and the automatically planned model can be obtained at this time. The mesh on the fitted resection surface supports direct editing, and also supports the adjustment of the size of the direct editing hot zone, which is used to adjust the editing range of the mesh on the resection surface during the editing process. When the resection surface generated by the above process does not meet the requirements, the position of the control point can be adjusted to control the initial direction of the surface, and a new surface planning result can be regenerated to adjust the resection direction of the resection surface; by supporting direct editing operations, the fitted resection surface can be intuitively dragged and edited without the need for a complex surface reconstruction process.

[0105] In some examples, the mouse cursor is moved to the part of the cut surface to be edited. At this time, the hot zone will be displayed in different colors to distinguish the area to be edited and the area that will not be edited on the cut surface. Use the scroll wheel to scroll up and down to adjust the editing area, then press the left button to pick up the surface model and determine the size of the hot zone. Finally, keep the left button pressed and move the cursor to edit the cut surface. After editing to the appropriate position, release the left mouse button to complete the editing; at this time, move the cursor to continue the next editing operation. It should be noted that the editing process of the cut surface is a step used to optimize the surface planning results, and it is not a necessary step.

[0106] In one of the embodiments, the first grid point is a grid point where the projection on the reference normal vector intersects with the target area; the second grid point is a grid point where the projection on the reference normal vector does not intersect with the target area.

[0107] Specifically, each grid point in the plane model can be projected onto the model to be processed based on the reference normal vector, wherein the first grid point is the grid point where the projection on the reference normal vector intersects with the target area; the second grid point is the grid point where the projection on the reference normal vector does not intersect with the target area, thereby enabling each grid point in the plane model to be divided into a plurality of first grid points and a plurality of second grid points. The above method can accurately divide each first grid point and each second grid point without manual designation, and has a high degree of accuracy. Based on the mapping of each first grid point and each second grid point obtained in the above method, the resection surface can be automatically generated, and the accuracy of surface planning is improved.

[0108] The method of this application can be applied to medical imaging data of any organ or any part. In one example, Fig. 9 As shown in (a), the lung organs and lesion tissues in the medical imaging data are segmented, and the segmentation results of the lung tissues are three-dimensionally reconstructed to obtain the corresponding reconstructed images, and then the model to be processed is generated. The surface area of ​​the model to be processed can correspond to the surface of the lung organs, wherein the target area includes the area to be segmented, and the target area is obtained by expanding the area to be segmented based on the preset safety boundary. It should be noted that the present application does not limit the segmentation algorithm and the model reconstruction algorithm, which are only used as the prerequisites of the present application. Fig. 9 As shown in (b), the resection surface generated by automatically determining the control points based on the above surface planning method is viewed from different perspectives; Fig. 9 As shown in (c) in FIG. 1 , the resection surface generated by the above surface planning method is shown in different viewing angles by specifying control points; Fig. 9 (d) is a schematic diagram of hot zone size adjustment at different viewing angles when the grid of the cut surface supports manual editing.

[0109] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0110] Based on the same inventive concept, the embodiment of the present application also provides a surface planning device for implementing the surface planning method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more surface planning device embodiments provided below can refer to the limitations of the surface planning method above, and will not be repeated here.

[0111] In one embodiment, Fig.10 As shown, a surface planning device is provided, the device comprising:

[0112] The to-be-processed model acquisition module 1010 is used to acquire the to-be-processed model corresponding to the medical image data based on the medical image data;

[0113] The plane model generation module 1020 is used to determine the reference points and control points of the model to be processed, and generate a plane model for the model to be processed according to the reference points and the control points; the reference points are located in the target area of ​​the model to be processed; the control points are located in the surface area of ​​the model to be processed;

[0114] The surface planning result acquisition module 1030 is used to map the plane model onto the model to be processed to obtain the surface planning result for the model to be processed; the surface planning result is used to indicate the path for cutting the target area from the model to be processed.

[0115] In one embodiment, the processing model acquisition module 1010 is also used to obtain segmentation results for the area to be segmented and the surface area based on the medical image data; based on the segmentation results and the preset safety boundary of the area to be segmented, obtain the processing model including the target area.

[0116] In one of the embodiments, the plane model generation module 1020 is also used to determine reference points based on the target area; determine control points based on the reference points and the surface area; process the reference points and the control points to determine a reference normal vector for the plane model; and generate a plane model based on the reference normal vector.

[0117] In one of the embodiments, the plane model generation module 1020 is also used to determine the center of gravity of the target area or the center of gravity of the bounding box of the target area as a reference point; determine the point on the surface area that is closest to the reference point as a control point; and determine the direction from the reference point to the control point as a reference normal vector for the plane model.

[0118] In one of the embodiments, the plane model generation module 1020 is also used to determine the initial plane where the plane model is located based on the reference normal vector; the initial plane is a plane where the control points deviate from the model to be processed by a preset distance along the reference normal vector; based on the bounding box for the target area, the boundary parameters of the plane model are determined; the initial plane is gridded, and the plane model is generated in combination with the boundary parameters of the plane model.

[0119] In one of the embodiments, the plane model generation module 1020 is further configured to enlarge the two-dimensional surface corresponding to the projection based on the projection of the bounding box of the target area on the initial plane, and determine the boundary parameters of the plane model.

[0120] In one of the embodiments, the surface planning result acquisition module 1030 is also used to divide each grid point in the plane model into multiple first grid points and multiple second grid points based on a reference normal vector; map each first grid point to a target area of ​​the model to be processed to obtain multiple target mapping points; the target mapping point is the farthest mapping point of the first grid point in the target area based on the reference normal vector; and fit the surface planning result based on each target mapping point and each second grid point.

[0121] Each module in the above-mentioned surface planning device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0122] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0123] In one embodiment, a surgical robot system is provided, the system comprising a surgical robot; the surgical robot is used to acquire medical image data and a model to be processed corresponding to the medical image data, and the surgical robot executes the above method.

[0124] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0125] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the above method when executed by a processor.

[0126] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.11As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a surface planning method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0127] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0128] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0129] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A surface planning method, characterized in that: The method comprises: Based on the medical image data, obtaining a model to be processed corresponding to the medical image data; Determine a reference point and a control point of the model to be processed, and generate a plane model for the model to be processed according to the reference point and the control point; the reference point is in a target area of ​​the model to be processed; the control point is in a surface area of ​​the model to be processed; The plane model is mapped onto the model to be processed to obtain a surface planning result for the model to be processed; the surface planning result includes a surface for cutting off the target area from the model to be processed.

2. The method according to claim 1, characterized in that The step of obtaining the to-be-processed model corresponding to the medical image data based on the medical image data includes: Based on the medical image data, obtaining segmentation results for the area to be segmented and the surface area; Based on the segmentation result and a preset safety boundary for the area to be segmented, the model to be processed including the target area is obtained.

3. The method according to claim 1, characterized in that The step of determining reference points and control points of the model to be processed, and generating a plane model for the model to be processed according to the reference points and the control points, comprises: Based on the target area, determining the reference point; Determining the control point based on the reference point and the surface area; Processing the reference point and the control point to determine a reference normal vector for the plane model; The plane model is generated according to the reference normal vector.

4. The method according to claim 3, characterized in that: The step of determining the reference point based on the target area includes: Determine the center of gravity of the target area or the center of gravity of the bounding box of the target area as the reference point; The step of determining the control point based on the reference point and the surface area comprises: Determine the point on the surface area that is closest to the reference point as the control point; The processing of the reference point and the control point to determine a reference normal vector for the plane model includes: The direction from the reference point to the control point is determined as a reference normal vector for the plane model.

5. The method according to claim 3, characterized in that: The step of generating the plane model according to the reference normal vector comprises: Determine the initial plane where the plane model is located according to the reference normal vector; the initial plane is the plane where the control point deviates from the model to be processed by a preset distance along the reference normal vector; Determining boundary parameters of the plane model based on a bounding box for the target area; The initial plane is meshed and the plane model is generated in combination with the boundary parameters of the plane model.

6. The method according to claim 5, characterized in that The step of determining the boundary parameters of the plane model based on the bounding box of the target area includes: Based on the projection of the bounding box of the target area on the initial plane, the two-dimensional surface corresponding to the projection is enlarged to determine the boundary parameters of the plane model.

7. The method according to claim 5, characterized in that The step of mapping the plane model onto the model to be processed to obtain a surface planning result for the model to be processed further includes: Based on the reference normal vector, each grid point in the plane model is divided into a plurality of first grid points and a plurality of second grid points; Mapping each of the first grid points into the target area of ​​the model to be processed to obtain a plurality of target mapping points; the target mapping point is the farthest mapping point of the first grid point in the target area based on the reference normal vector; The surface planning result is obtained by fitting according to each of the target mapping points and each of the second grid points.

8. The method according to claim 7, characterized in that The first grid points are grid points where the projection of the reference normal vector on the target area intersects; the second grid points are grid points where the projection of the reference normal vector on the target area does not intersect.

9. A surface planning device, characterized in that: The device comprises: A model-to-be-processed acquisition module, used for acquiring a model-to-be-processed corresponding to the medical image data based on the medical image data; A plane model generation module, used to determine the reference point and the control point of the model to be processed, and generate a plane model for the model to be processed according to the reference point and the control point; the reference point is in the target area of ​​the model to be processed; the control point is in the surface area of ​​the model to be processed; The surface planning result acquisition module is used to map the plane model onto the model to be processed to obtain the surface planning result for the model to be processed; the surface planning result is used to indicate the path for cutting the target area from the model to be processed.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A surgical robot system, characterized in that: The system includes a surgical robot; the surgical robot is used to acquire medical image data and acquire a model to be processed corresponding to the medical image data, and the surgical robot executes the method described in any one of claims 1 to 8.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images