Visualization method and device for femur cutting, medium and product

Abstract

The invention discloses a visualization method and device for thighbone cutting, a medium and a product, and relates to the technical field of medical image.The method comprises the steps that a thighbone model to be reconstructed is visualized, and cutting surface selection parameters input by a user are obtained, extracting virtual cutting surface grid data from pre-acquired to-be-reconstructed thighbone grid data based on the cutting surface selection parameters, extracting regional grid data corresponding to different cutting ranges from the to-be-reconstructed thighbone grid data based on the virtual cutting surface grid data, and reconstructing the to-be-reconstructed thighbone grid data based on the regional grid data. And generating and caching rendering data of each region, obtaining pose data of the surgical instrument, and performing cutting rendering in the to-be-reconstructed thighbone model based on the pose data and the rendering data of the region corresponding to the pose data. According to the method, the region rendering data of each cutting range is pre-generated and cached before the operation, and the rendering data can be directly read for visualization during the subsequent cutting operation, so that the visual operation efficiency of femoral cutting is improved.

Description

Technical Field

[0001] This application relates to the field of medical image technology, and particularly to a visualization method, device, medium, and product for femoral cutting. Background Art

[0002] In navigated knee replacement surgery, femoral condyle cutting is a key operation step, and its visual presentation has a decisive impact on the surgical accuracy. The traditional visualization scheme for femoral condyle cutting is as follows: converting the surface mesh data of the femoral condyle into voxelized three-dimensional model data, then simulating the cutting process through Boolean operations, and then converting the operation result back into surface mesh data, and then performing rendering processing on the converted surface mesh data.

[0003] However, in the above scheme, each time a cutting operation is performed, it is necessary to convert the voxelized three-dimensional model data after operation into surface mesh data, and after calculating the model data after cutting, perform rendering processing on the model, resulting in low visualization operation efficiency. Summary of the Invention

[0004] This application provides a visualization method, device, medium, and product for femoral cutting, which can improve the visualization operation efficiency of femoral condyle cutting.

[0005] To achieve the above object, this application proposes a visualization method for femoral cutting, including:

[0006] Visualize the femoral model to be reconstructed;

[0007] Obtain the cutting plane selection parameters input by the user, and extract virtual cutting plane mesh data from the pre-obtained femoral mesh data to be reconstructed based on the cutting plane selection parameters;

[0008] Based on the virtual cutting plane mesh data, extract regional mesh data corresponding to different cutting ranges from the femoral mesh data to be reconstructed;

[0009] Generate and cache the rendering data for each region based on each piece of the regional mesh data;

[0010] Obtain the pose data of the surgical instrument, and perform cutting rendering in the femoral model to be reconstructed based on the pose data and the rendering data of the region corresponding to the pose data.

[0011] In addition, to achieve the above object, this application also proposes a terminal device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the visualization method for femoral cutting as described above.

[0012] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the visualization method for femoral cutting as described above are implemented.

[0013] In addition, to achieve the above object, the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the visualization method for femoral cutting as described above are implemented.

[0014] One or more technical solutions proposed by the present application have at least the following technical effects:

[0015] By visualizing the femur model to be reconstructed, obtaining the cutting plane selection parameters input by the user, extracting the virtual cutting plane mesh data from the pre-obtained femur mesh data to be reconstructed based on the cutting plane selection parameters, extracting the regional mesh data corresponding to different cutting ranges from the femur mesh data to be reconstructed based on the virtual cutting plane mesh data, and generating and caching the rendering data of each region based on the regional mesh data, obtaining the pose data of the surgical instrument, and performing cutting rendering in the femur model to be reconstructed based on the pose data and the rendering data of the region corresponding to the pose data. In this embodiment, the surface mesh data is directly used for cutting simulation without format conversion, which improves the cutting operation efficiency. In addition, by pre-generating and caching the regional rendering data of each cutting range before surgery, thus, during subsequent cutting operations, the rendering data can be directly read to perform cutting visualization on the femur model, achieving the cutting visualization efficiency. In summary, this embodiment improves the visualization operation efficiency of femoral condyle cutting. Description of the Drawings

[0016] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0017] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is an application scenario diagram for femoral cutting related to the present application;

[0019] Figure 2 It is a schematic flowchart provided by an embodiment of the visualization method for femoral cutting of the present application;

[0020] Figure 3Visualization effect diagram of the original femoral model provided by the embodiment of the present application;

[0021] Figure 4 Visualization effect diagram of the femoral model to be reconstructed provided by the embodiment of the present application;

[0022] Figure 5 Effect diagram of cutting and rendering the distal femoral condyle selected in the femoral model to be reconstructed provided by the embodiment of the present application;

[0023] Figure 6 Effect diagram of the cutting boundary area and the cutting buffer area provided by the embodiment of the present application;

[0024] Figure 7 Display effect of the rendering layers of each area provided by the embodiment of the present application;

[0025] Figure 8 Effect diagram of the femoral model shown when the posterior condyle cutting is selected provided by the present application;

[0026] Figure 9 Structural schematic diagram of the terminal device provided by the embodiment of the present application.

[0027] The realization of the purpose, functional features and advantages of the present application will be further described in conjunction with the embodiments with reference to the accompanying drawings. Specific Embodiments

[0028] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0029] In order to better understand the technical solutions of the present application, the following will be described in detail in conjunction with the accompanying drawings of the specification and specific embodiments.

[0030] Femoral condyle cutting is an important step in navigated knee replacement surgery, and its main steps are as follows:

[0031] As Figure 1 shown, Figure 1The figure shows an application scenario diagram of femoral cutting involved in the present application. Before surgery, in the image space (under the image coordinate system Fimage), the user moves the femoral prosthesis model in the interface and places it at a reasonable position on the patient's femoral model. The area where the femoral prosthesis model overlaps with the patient's femoral model is the part of the femoral condyle that needs to be cut and removed during the operation. During the operation, an optical tracking array is rigidly connected to the surgical instrument and the patient's femur respectively. The positioning camera can transmit in real time the transformation matrix TcameraToToolRF from the positioning camera coordinate system Fcamera to the instrument tracking array coordinate system FtoolRF, and the transformation matrix TcameraToBoneRF from Fcamera to the bone tracking array coordinate system FboneRF. After the image registration operation, the above two matrices can be used for image navigation and positioning. The final effect is that the relative position relationship between the patient's femur and the cutting instrument in the real world conforms to the relative position relationship between the femoral model and the surgical instrument model in the image space. When moving the real patient's femur or the real surgical instrument, the relative position relationship between the femoral model and the surgical instrument model in the image will also move accordingly.

[0032] Then, guided by the image, move the cutting tool to the area to be cut. During the process of cutting with the surgical instrument, as the real bone is gradually cut and removed, the femoral model in the image space also needs to deform accordingly until the part to be cut ( Figure 1 the green part in) is completely removed. The above process involves real-time cutting operation of the femoral model and visualization of the cutting result.

[0033] The traditional femoral condyle cutting visualization scheme is: convert the surface mesh data of the femoral condyle into voxelized three-dimensional model data, then simulate the cutting process through Boolean operations, then convert the operation result back to surface mesh data, and then perform rendering processing on the converted surface mesh data.

[0034] However, in the above scheme, during each cutting operation, it is necessary to convert the voxelized three-dimensional model data after operation into surface mesh data, and after calculating the model data after cutting, perform rendering processing on the model, resulting in low visualization operation efficiency.

[0035] Based on this, the embodiment of the present application provides a femoral cutting visualization method. The execution subject of this embodiment can be a terminal device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc. Hereinafter, taking the terminal device as an example, this embodiment and the following embodiments will be described.

[0036] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of an embodiment of the femoral cutting visualization method of the present application.

[0037] In this embodiment, the femoral cutting visualization method includes steps S10 to S40:

[0038] Step S10, visualize the femoral model to be reconstructed.

[0039] Specifically, the femoral model to be reconstructed includes an original femoral model and a femoral prosthesis model with a position association relationship. The terminal device can obtain the original femoral model data and the femoral prosthesis model data. Both the original femoral model data and the femoral prosthesis model data are surface mesh data (hereinafter referred to as mesh data), and the original femoral model mesh data and the femoral prosthesis model mesh data are pre-generated and stored. Then, the terminal device performs rendering processing on the original femoral mesh data and the femoral prosthesis mesh data on a preset rendering platform to render the original femoral model and the femoral prosthesis model in the display interface, thereby realizing the visualization of the femoral model to be reconstructed. Among them, the preset rendering platform can adopt the Vtk (visualization toolkit) image rendering and display architecture. Vtk is an open-source and free software system mainly used for three-dimensional computer graphics, image processing, and visualization.

[0040] Please refer to Figure 3 , Figure 3 which is the visualization effect diagram of the original femoral model provided by this application, Figure 4 and Figure 4 is the visualization effect diagram of the femoral model to be reconstructed provided by this application. Among them, Figure 4 (a) in

[0041] is the effect diagram of the femoral prosthesis model, and

[0042] (b) in

[0043] is the effect diagram of the femoral prosthesis model embedded in the original femoral model. Step S20, obtain the cutting plane selection parameters input by the user, and extract the virtual cutting plane mesh data from the pre-obtained femoral mesh data to be reconstructed based on the cutting plane selection parameters.

[0042] It should be noted that due to the physiological structure of the femur, the femoral cutting area is generally defined by 5 planes, including the anterior condyle, anterior oblique, distal, posterior oblique, and posterior condyle, and the angles between the sections are different. Therefore, in specific implementation, the user needs to select the cutting plane. Then, the terminal device extracts the cutting plane selection parameters from the cutting plane selection instruction in response to the cutting plane selection instruction input by the user. The cutting plane selection parameters at least include the identifier of the cutting plane selected by the user for cutting, so that the terminal device can extract the mesh data belonging to the corresponding cutting plane from the femoral mesh data to be reconstructed according to the cutting plane identifier to obtain the virtual cutting plane mesh data.

[0043] In some embodiments, the terminal device can perform a Boolean operation on the femoral mesh data to be reconstructed and the virtual cutting plane mesh data in MeshLib to extract the regional mesh data corresponding to different cutting ranges.

[0044] It should be noted that MeshLib is an open-source library for processing and operating three-dimensional mesh data structures. It provides tools such as mesh data identification, transformation, simplification, subdivision, and Boolean operations, and supports common mesh data file formats such as.stl and.ply. The data type of the mesh data in MeshLib is MRMesh. Since MeshLib supports efficient Boolean operations, the calculation efficiency can be improved.

[0045] Step S30: Based on the virtual cutting plane mesh data, extract the regional mesh data corresponding to different cutting ranges from the femoral mesh data to be reconstructed, and generate and cache the rendering data for each region based on the regional mesh data.

[0046] It is worth noting that in orthopedic surgical operations, doctors need to use surgical instruments to precisely cut the femur, and the corresponding anatomical structures of the femur to be reconstructed will be partially removed due to different cutting ranges. To improve surgical safety and operation accuracy, multiple sets of cutting range plans can be preset for different clinical scenarios in the preoperative planning stage, and each region is discretized into a set of mesh units that can be independently rendered and cached. When performing a specific cutting operation during the operation, by locating the region where the cutting operation is located and using the pre-cached rendering data of the corresponding region to render the model. Thus, for each subsequent cutting operation, it is not necessary to re-render the mesh data of the corresponding region for different cutting operations, improving the visualization operation efficiency of femoral condyle cutting.

[0047] In some embodiments, the terminal device can extract the mesh data within the preset planned cutting range from the femoral mesh data to be reconstructed to obtain the mesh data of the area to be polished, and extract from the femoral mesh data to be reconstructed the mesh data that exceeds the preset planned cutting range but does not exceed the preset buffer cutting range, the mesh data that exceeds the preset buffer cutting range and forms a closed geometric body, and the mesh data that exceeds the preset buffer cutting range, respectively obtaining the mesh data of the buffer area, the warning area, and the thin shell area. Among them, the preset planned cutting range and the preset buffer cutting range are set according to the actual situation. By extracting the mesh data of the area to be polished, the buffer area, the warning area, and the thin shell area, different levels of cutting ranges can be established, enabling the operator to quickly locate the effective working range and the warning working range. After the terminal device extracts the mesh data of each region, it can configure an identifier for each region.

[0048] In some embodiments, generating and caching the rendering data for each region based on the grid data of each region includes: processing the grid data of each region according to a preset rendering rule in a preset rendering platform to generate the rendering data for each region, where the rendering effects presented by the rendering data of different regions are different; then, caching the rendering data for each region.

[0049] After the terminal device generates the rendering data for each region, it can generate and store the correspondence between the region identifier and the rendering data, so that the rendering data for the corresponding region can be directly read based on the region identifier subsequently.

[0050] Step S40, obtaining the pose data of the surgical instrument, and performing cutting rendering on the femur model to be reconstructed based on the pose data and the rendering data of the region corresponding to the pose data.

[0051] It should be noted that the pose data of the surgical instrument is data in the positioning camera space, and the femur model data to be reconstructed is data in the image space. Therefore, in specific implementation, after the terminal device obtains the pose data of the surgical instrument, it needs to convert the pose data into the image space coordinate system, and then determine the spatial position of the saw blade model in the femur model to be reconstructed according to the converted pose data. Since it is a prior art, it will not be elaborated here.

[0052] In some embodiments, the terminal device can convert the pose data of the surgical instrument into the coordinate space where the femur model to be reconstructed is located, obtain the spatial position of the pose data mapped into the femur model to be reconstructed, then determine the region where the spatial position is located, and render the saw blade model of the surgical instrument and render the region based on the rendering data of the corresponding region at the first time node to obtain the femur model to be reconstructed with the corresponding region rendering layer superimposed, so as to visually display which cutting range region the surgical instrument is in when it moves to this position, to remind the operator whether it is within the safe cutting range. Then, based on the femur model to be reconstructed, combining the saw blade model grid data and the spatial position where it is located for cutting operation, obtaining the grid data of the cut femur model, and rendering the cut femur model based on the femur model to be reconstructed with the corresponding region rendering layer superimposed and the grid data of the cut femur model at the second time node, where the second time node is later than the first time node.

[0053] Please refer to Figure 5 , Figure 5 For the rendering effect diagram of performing cutting rendering on the femur model to be reconstructed when selecting distal femoral condyle cutting. Figure 5 In (a) shows the rendering effect display diagram when the saw blade model moves to the femur model to be reconstructed, Figure 5 In (b) shows the rendering effect display diagram after the saw blade model cuts the femur model. Figure 5The blue rectangle in (a) represents the saw blade model, and the green part represents the area to be polished, that is, the area to be cut. Figure 5 As shown in (a) in the figure, when the saw blade model moves to the corresponding position along the planned path, the three-dimensional shape of the resected bone tissue (the green area in the figure gradually shows the cutting effect), the saw blade model and the rendering layer of the area where it is located (the rendering layer of the saw blade model is above the rendering layer of the area where it is located). Figure 5 In (b), when the cutting operation is completed, the bone tissue at the location of the saw blade model has been partially removed, and the original red rendering area (i.e., the red rendering layer) has also been reduced accordingly.

[0054] This embodiment obtains the cutting surface selection parameters input by the user by visualizing the femoral model to be reconstructed, and extracts virtual cutting surface mesh data from the pre-acquired femoral mesh data to be reconstructed based on the cutting surface selection parameters, extracts regional mesh data corresponding to different cutting ranges from the femoral mesh data to be reconstructed based on the virtual cutting surface mesh data, and generates and caches rendering data of each region based on each regional mesh data, obtains the posture data of the surgical instrument, and performs cutting rendering in the femoral model to be reconstructed based on the posture data and the rendering data of the region corresponding to the posture data. This embodiment directly uses surface mesh data for cutting simulation without format conversion, thereby improving the cutting operation efficiency. In addition, by pre-generating and caching the regional rendering data of each cutting range before the operation, the rendering data can be directly read to visualize the cutting of the femur during the subsequent cutting operation, thereby realizing instant visualization of the cutting. In summary, this embodiment improves the visualization operation efficiency of femoral condyle cutting.

[0055] In an optional implementation, the above S40 may include S401 to S404:

[0056] S401, obtaining a pre-generated saw blade model, and determining the spatial position of the saw blade model in the femoral model to be reconstructed based on the posture data, and determining the area where the spatial position is located to obtain the area corresponding to the posture data.

[0057] It should be noted that the saw blade model is a pre-established virtual saw blade model that is similar in appearance to the cutting saw blade of an actual surgical instrument.

[0058] After determining the spatial position of the saw blade model in the femoral model to be reconstructed, the terminal device determines the area where the spatial position is located to obtain which cutting range area of the femoral model to be reconstructed is mapped when the surgical instrument moves to this position.

[0059] S402, at a first time node, generating a rendering layer of the corresponding area based on the rendering data of the corresponding area and superimposing it on the femur model to be reconstructed, obtaining the femur model to be reconstructed superimposed with the rendering layer of the corresponding area, and rendering a saw blade model at the corresponding spatial position.

[0060] In addition, the terminal device can, at the first time node after obtaining the pose data, obtain the rendering data of the corresponding area from the cache based on the identifier of the corresponding area, generate a rendering layer of the corresponding area based on the rendering data of the corresponding area, and superimpose the rendering layer of the corresponding area on the visualized femoral model to be reconstructed, so as to remind the user that if the surgical instrument moves to this position, this part of the area will be cut.

[0061] S403. Perform a cutting operation on the femoral model mesh data to be reconstructed based on the spatial position and the saw blade model mesh data to obtain the cut femoral model mesh data.

[0062] After the terminal device determines the spatial position of the saw blade model in the femoral model to be reconstructed, it can calculate the corresponding saw blade model mesh data point set when the saw blade model is in this spatial position based on the spatial position and the saw blade model mesh data, and then perform a Boolean operation of subtracting the saw blade area mesh data point set from the femoral model mesh data point set to be reconstructed in MeshLib to obtain the cut femoral model mesh data.

[0063] S404. Render the cut femoral model based on the femoral model to be reconstructed with the rendering layer of the corresponding area superimposed and the cut femoral model mesh data at the second time node, where the second time node is later than the first time node.

[0064] After the terminal device determines the cut femoral model mesh data, it can select the cut femoral model mesh data for visualization display on the basis of the femoral model to be reconstructed with the rendering layer of the corresponding area superimposed, so as to render the cut femoral model and display it to the user side.

[0065] In a feasible implementation manner, the above S20, extracting the area mesh data corresponding to different cutting ranges from the femoral mesh data to be reconstructed based on the virtual cutting plane mesh data, may include:

[0066] S201. Generate cutting boundary area mesh data and cutting buffer area mesh data based on the virtual cutting plane mesh data, where the cutting boundary area is generated by the virtual cutting plane extending unidirectionally along the normal direction, and the bottom surface of the cutting boundary area coincides with the virtual cutting plane, the cutting buffer area is generated by the virtual cutting plane extending bidirectionally along the normal direction, and the bottom surface of the cutting buffer area is parallel to the virtual cutting plane, and the normal distance between the bottom surface of the cutting buffer area and the virtual cutting plane is a preset buffer distance.

[0067] Specifically, the cutting boundary region is generated by the one-way extension of the virtual cutting plane along the normal direction, and the cutting buffer region is generated by the two-way extension of the virtual cutting plane along the normal direction. The sinking distance from the virtual cutting plane is n, where n is a preset buffer distance, which represents the thickness of the buffer region. The bottom surface and thickness of the mesh data of the cutting boundary region and the mesh data of the cutting buffer region need to cover all the regions to be cut. Please refer to Figure 6 , Figure 6 which shows the effect diagrams of the cutting boundary region and the cutting buffer region. Figure 6 In it, a1 is the virtual cutting plane, a2 is the cutting boundary region, and a3 is the cutting buffer region.

[0068] S202. Perform a Boolean operation on the mesh data of the cutting boundary region and the mesh data of the femur to be reconstructed, so as to extract the mesh data within the preset planned cutting range from the mesh data of the femur to be reconstructed, and obtain the mesh data of the area to be polished.

[0069] Specifically, the terminal device can perform a Boolean operation of intersection on the mesh data of the cutting boundary region and the mesh data of the femur to be reconstructed in MeshLib, that is, use the MR::BooleanOperation::Intersection method of MeshLib, with the mesh data of the femur to be reconstructed bone_mesh and the mesh data of the cutting boundary region cut_mesh as inputs, obtain the intersection of the two, and denote it as green_mesh, so as to extract the mesh data within the preset planned cutting range from the mesh data of the femur to be reconstructed, and obtain the mesh data of the area to be polished.

[0070] S203. Perform a Boolean operation based on the mesh data of the cutting boundary region, the mesh data of the cutting buffer region, and the mesh data of the femur to be reconstructed, so as to extract from the mesh data of the femur to be reconstructed the mesh data that exceeds the preset planned cutting range but does not exceed the preset buffer cutting range, the mesh data that exceeds the preset buffer cutting range and forms a closed geometric body, and the mesh data that exceeds the preset buffer cutting range, and correspondingly obtain the mesh data of the buffer region, the mesh data of the warning region, and the mesh data of the thin shell region.

[0071] The terminal device can perform a Boolean operation of intersection on the mesh data of the cutting buffer region and the mesh data of the femur to be reconstructed, obtaining the intersecting part of the two as the intersecting part mesh data, and then perform a Boolean operation of subtraction on the cutting boundary region data and the intersecting part mesh data to obtain the buffer region mesh data. Specifically, the terminal device can use the MR::BooleanOperation::Intersection method of MeshLib, taking the mesh data of the femur to be reconstructed bone_mesh and the mesh data of the cutting buffer region cutPlus_mesh as inputs, obtaining the intersecting part of the two, denoted as bufferTmp_mesh, and then using the MR::BooleanOperation::DifferenceAB method of MeshLib, taking bufferTmp_mesh and the mesh data of the cutting boundary region cut_mesh as inputs, obtaining the part of bufferTmp_mesh outside cut_mesh, which is the buffer region mesh data, and denoting it as buffer_mesh.

[0072] The terminal device can perform a subtraction Boolean operation of the first type on the mesh data of the cutting buffer region and the mesh data of the femur to be reconstructed, obtaining the warning region mesh data. Among them, the figure or geometric body obtained after the subtraction operation of the first type is closed. Specifically, the terminal device can use the MR::BooleanOperation::DifferenceAB method of MeshLib, taking bone_mesh and cutPlus_mesh as inputs, obtaining the part of bone_mesh outside cutPlus_mesh, denoted as red_mesh, and red_mesh is the warning region mesh data.

[0073] The terminal device can perform a subtraction Boolean operation of the second type on the mesh data of the cutting buffer region and the mesh data of the femur to be reconstructed, obtaining the thin shell region mesh data. Among them, the figure or geometric body obtained after the subtraction operation of the second type is an unclosed figure. Specifically, the terminal device can use the MR::BooleanOperation::OutsideA method of MeshLib, taking bone_mesh and cutPlus_mesh as inputs, obtaining the unclosed part of bone_mesh outside cutPlus_mesh, denoted as shell_mesh, and shell_mesh is the thin shell region mesh data.

[0074] In a feasible implementation manner, after extracting the virtual cutting surface mesh data from the pre-obtained femur mesh data to be reconstructed based on the cutting surface selection parameter, it further includes: rendering a virtual cutting surface in the femur model to be reconstructed based on the virtual cutting surface mesh data.

[0075] By rendering the virtual cutting plane grid data, the cutting plane can be visually presented to the operator, facilitating the operator to cut the femur to be reconstructed based on the rendered virtual cutting plane.

[0076] In a feasible implementation manner, after the above S40, steps S50 to S80 are further included:

[0077] S50, visualize the cut femur model obtained after cutting the femur model to be reconstructed, and record the grid data of the cut femur.

[0078] Specifically, the cut femur model is displayed according to the default display effect, that is, the cut femur model does not include the rendering layer of the corresponding area superimposed during the previous cutting operation.

[0079] S60, obtain the cutting plane replacement parameters input by the user, and extract the reselected virtual cutting plane grid data from the grid data of the cut femur based on the cutting plane replacement parameters.

[0080] Since there are 5 femur cutting areas, after the first cut, it may be necessary to select a new cutting plane for cutting. At this time, the terminal device can receive the cutting plane replacement instruction input by the user, extract the cutting plane replacement parameters from the cutting plane replacement instruction, and then extract the reselected virtual cutting plane grid data from the grid data of the cut femur based on the cutting plane replacement parameters.

[0081] S70, perform a Boolean operation based on the reselected virtual cutting plane grid data and the grid data of the cut femur to extract the regional grid data corresponding to different cutting ranges from the cut femur model, and generate and cache the rendering data for each region of the cut femur model based on the regional grid data extracted from the cut femur model

[0082] Specifically, the specific implementation steps and principles of this step S70 are similar to those of the above step S30, and will not be elaborated here.

[0083] S80, obtain the pose data of the surgical instrument after reselecting the cutting plane, and perform cutting rendering in the cut femur model based on the pose data after reselecting the cutting plane and the rendering data of its corresponding region.

[0084] Specifically, the specific implementation steps and principles of this step S80 are similar to those of the above step S40, and will not be elaborated here.

[0085] The terminal device can repeat the above steps S50 to S80 to select other cutting planes for cutting operation and cutting visualization until all cutting operations and visualizations of all cutting planes are completed.

[0086] Exemplarily, to facilitate the understanding of the visualization method for femoral cutting in this embodiment, the following presents a specific implementation process in an application scenario, including:

[0087] A1. Obtain the original femoral mesh data and denote it as bone_mesh, save bone_mesh to a bone file, obtain the femoral prosthesis mesh data and denote it as implant_mesh, obtain the saw blade model mesh data blade_mesh, and save blade_mesh to a blade file.

[0088] A2. Convert the original femoral mesh data bone_mesh and the femoral prosthesis mesh data implant_mesh into data in vtkPolyData format. The format conversion steps are as follows:

[0089] A2-a. Use the points.vec method of MRMesh to obtain a floating-point number array denoted as P1. Each element in this array contains the coordinate values of each vertex of the mesh data, arranged in the order of x0, y0, z0, x1, y1, z1, x2, y2, z2, ….

[0090] A2-b. Create a new point set data of vtkPoints type denoted as P2, and set the coordinate values of each point in P1 into P2.

[0091] A2-c. Use the topology.getAllTriVerts() method of MRMesh to obtain an integer array denoted as P3. Every 3 elements in this array form a group, recording which vertices constitute each triangular mesh of the mesh data. For example, if P3 is 3, 1, 4, 0, 5, 2, it means that this MRMesh mesh data contains two triangular meshes. The first triangular mesh is formed by connecting vertices 3, 1, and 4, and the second triangular mesh is formed by connecting vertices 0, 5, and 2.

[0092] A2-d. Create a vtkTriangle for every three elements in P3 as a group, and then store all vtkTriangles into a newly created vtkCellArray, and denote this vtkCellArray as P4.

[0093] A2-e. Create a new vtkPolyData, use the SetPoints() method to set P2 as its vertices, and use its SetPolys() method to set P4 as its surface triangular patches, that is, convert the data of MRMesh type into vtkPolyData format.

[0094] B. Visualize the femoral model to be reconstructed. The femoral model to be reconstructed includes the original femoral model and the femoral prosthesis model with a position association relationship.

[0095] Visualize the femur model to be reconstructed in the display interface through the original femur model data and the femoral prosthesis model data converted into the vtkPolyData format. Among them, the original femur model is displayed in white, and the femoral prosthesis model is displayed in green, as Figure 4 shown.

[0096] C. According to the cutting plane selected by the user, extract the virtual cutting plane from the femoral prosthesis grid data, and then generate columnar cutting boundary region grid data cut_mesh and cutting buffer region cutPlus_mesh based on the virtual cutting plane. The bottom surface of the cutting boundary region coincides with the virtual cutting plane, the bottom surface of the cutting buffer region is parallel to the virtual cutting plane, and the sinking distance from the virtual cutting plane is n. The value of n is the thickness of the white buffer region specified by the user. The bottom area and thickness of the cutting boundary region and cutPlus need to be large enough to ensure that all the bone regions to be cut are surrounded. Then, save the cutting boundary region grid data cut_mesh to the cut file, and save the cutting buffer region cutPlus_mesh to the cutPlus file.

[0097] D. Read the bone file and the cut file. Using the MR::BooleanOperation::Intersection method of MeshLib, with bone_mesh and cut_mesh as inputs, obtain the area to be polished and denote it as green_mesh. Using the MR::BooleanOperation::Intersection method of MeshLib, with bone_mesh and cutPlus_mesh as inputs, obtain the intersection of the two and denote it as bufferTmp_mesh; using the MR::BooleanOperation::DifferenceAB method of MeshLib, with bufferTmp_mesh and cut_mesh as inputs, obtain the part of bufferTmp_mesh that falls outside cut_mesh, which is the mesh data of the buffer area and denote it as buffer_mesh. Using the MR::BooleanOperation::DifferenceAB method of MeshLib, with bone_mesh and cutPlus_mesh as inputs, obtain the part of bone_mesh that falls outside cutPlus_mesh and denote it as red_mesh. Using the MR::BooleanOperation::OutsideA method of MeshLib, with bone_mesh and cutPlus_mesh as inputs, obtain the unclosed part of bone_mesh that falls outside cutPlus_mesh (retain the holes after cutting to expose the red part inside), that is, obtain the mesh data of the thin shell area and denote it as shell_mesh.

[0098] E. Generate the rendering data for each area

[0099] Using VTK, convert green_mesh, buffer_mesh, red_mesh, and shell_mesh into vtkPolyData data for visualization and denote them as green_poly, buffer_poly, red_poly, and shell_poly respectively. Then, assign the above vtkPolyData to different vtkActors respectively, and use the VTK pipeline to render each vtkPolyData simultaneously in the same vtkRenderer. Set the colors of the vtkActors corresponding to buffer_poly and shell_poly to white, the color of the vtkActor corresponding to green_poly to green, and the color of the vtkActor corresponding to red_poly to red, and ensure that the coating corresponding to shell_poly is above red_poly to obtain the rendering data of green_mesh, buffer_mesh, red_mesh, and shell_mesh, that is, obtain the rendering layers corresponding to each region. Finally, the display effects of the rendering layers of each region are as Figure 7 shown.

[0100] F. Femoral cutting visualization operation, including:

[0101] Before the first cutting operation and visualization, read the blade file, generate vtkPolyData data for visualization and MRMesh data for cutting calculation respectively, and denote them as blade_poly and blade_mesh respectively.

[0102] The pose data (4x4 affine matrix) of the saw blade in the surgical instrument from the navigation system is respectively stored in a newly created vtkMatrix4x4 and MR::AffineXf3f data. The vtkTransform and the transform() method of MRMesh are respectively used to move blade_poly and blade_mesh by using the newly created vtkMatrix4x4 and MR::AffineXf3f data. Then, the vtkActor corresponding to blade_poly is updated to complete the movement of the saw blade in the image space, so as to determine the spatial position of the saw blade in the femoral model to be reconstructed, and the saw blade model is rendered at the corresponding spatial position, and the rendering data of the corresponding area is read to generate a rendering layer for the corresponding area and superimposed on the femoral model to be reconstructed. Then, using the MR::BooleanOperation::DifferenceAB method of MeshLib, the moved blade_mesh is used to cut bone_mesh, green_mesh, buffer_mesh, and red_mesh respectively, and the cutting results are saved in the original MRMesh data. Using the MR::BooleanOperation::OutsideA method of MeshLib, the moved reamer_mesh is used to cut shell_mesh, and the cutting results are still saved in the original shell_mesh data.

[0103] Finally, the cut femoral model is visualized according to the cutting results and the femoral model to be reconstructed with the corresponding area rendering layer superimposed.

[0104] G. Select another cutting plane for cutting operation and visualization, including:

[0105] The cut femoral model is displayed on the display interface. Among them, the cut femoral model is displayed with the default display effect (such as white), that is, only the shape of the cut femoral model is retained, and the rendering layer left after the previous cutting operation is not retained. When the user hopes to switch to the femoral condyle cutting, in response to the cutting plane replacement parameter input by the user, a virtual cutting plane corresponding to the condyle is rendered in the cut femoral model, as Figure 8 shown, Figure 8 is the effect diagram of the femoral model shown when the femoral condyle cutting is selected provided by this application.

[0106] Then, perform the above step F to continue the cutting visualization of the femoral condyle.

[0107] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the femoral cutting visualization method of this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0108] The present application provides a terminal device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the visualization method for femoral cutting in the above embodiments.

[0109] Reference is made below Figure 9 , which shows a schematic structural diagram of a terminal device suitable for implementing the embodiments of the present application. The terminal device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions: tablet computers), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 9 The terminal device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0110] As Figure 9 shown, the terminal device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the terminal device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the terminal device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a terminal device with various systems, it should be understood that it is not required to implement or include all the shown systems. More or fewer systems may be implemented or included alternatively.

[0111] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium. The computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by a processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in the present application are executed.

[0112] The terminal device provided by the present application adopts the visualization method for femoral cutting in the above-mentioned embodiments, and can solve the technical problem of low visualization operation efficiency in femoral condyle cutting. Compared with the prior art, the beneficial effects of the terminal device provided by the present application are the same as those of the visualization method for femoral cutting provided by the above-mentioned embodiments, and other technical features in the terminal device are the same as those disclosed in the method of the previous embodiment, and will not be elaborated herein.

[0113] It should be understood that each part disclosed in the present application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0114] As described above, the above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0115] The present application provides a computer-readable storage medium, having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the visualization method for femoral cutting in the above-mentioned embodiments.

[0116] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0117] The above computer-readable storage medium can be included in the terminal device; or it can exist independently without being assembled into the terminal device.

[0118] The computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0119] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0120] The modules involved in the embodiments described in the present application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself in some cases.

[0121] The readable storage medium provided by the present application is a computer-readable storage medium, and the computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned visualization method for femoral cutting, which can solve the technical problem of low visualization operation efficiency of femoral condyle cutting. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the visualization method for femoral cutting provided by the above embodiments, and will not be elaborated here.

[0122] The present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the visualization method for femoral cutting as described above.

[0123] The computer program product provided by the present application can solve the technical problem of low visualization operation efficiency of femoral condyle cutting. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the visualization method for femoral cutting provided by the above embodiments, and will not be elaborated here.

[0124] The above are only some embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural transformation made under the technical concept of the present application by using the content of the specification and drawings of the present application, or any direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A visualization method for femoral cutting, characterized in that, The method includes: Visualizing the femur model to be reconstructed; Obtaining the cutting plane selection parameters input by the user, and extracting virtual cutting plane mesh data from the pre-obtained femur mesh data to be reconstructed based on the cutting plane selection parameters; Based on the virtual cutting plane mesh data, extracting regional mesh data corresponding to different cutting ranges from the femur mesh data to be reconstructed; Generating and caching the rendering data of each region based on each piece of the regional mesh data; Obtaining the pose data of the surgical instrument, and performing cutting rendering in the femur model to be reconstructed based on the pose data and the rendering data of the region corresponding to the pose data.

2. The visualization method for femoral cutting according to claim 1, wherein The extracting regional mesh data corresponding to different cutting ranges from the femur mesh data to be reconstructed based on the virtual cutting plane mesh data includes: Generating cutting boundary region mesh data and cutting buffer region mesh data based on the virtual cutting plane mesh data, wherein the cutting boundary region is generated by the virtual cutting plane extending unidirectionally along the normal direction, and the bottom surface of the cutting boundary region coincides with the virtual cutting plane, and the cutting buffer region is generated by the virtual cutting plane extending bidirectionally along the normal direction, and the normal distance between the bottom surface of the cutting buffer region and the virtual cutting plane is a preset buffer distance; Performing a Boolean operation on the cutting boundary region mesh data and the femur mesh data to be reconstructed to extract the mesh data within the preset planned cutting range from the femur mesh data to be reconstructed, obtaining the mesh data of the area to be polished; Performing a Boolean operation based on the cutting boundary region mesh data, the cutting buffer region mesh data and the femur mesh data to be reconstructed to respectively extract the mesh data that exceeds the preset planned cutting range but does not exceed the preset buffer cutting range, the mesh data that exceeds the preset buffer cutting range and forms a closed geometric body, and the mesh data that exceeds the preset buffer cutting range from the femur mesh data to be reconstructed, and correspondingly obtaining buffer region mesh data, warning region mesh data and thin shell region mesh data.

3. The visualization method for femoral cutting according to claim 1, characterized in that, The extracting regional mesh data corresponding to different cutting ranges from the femur mesh data to be reconstructed based on the virtual cutting plane mesh data includes: Performing a Boolean operation on the femur mesh data to be reconstructed and the virtual cutting plane mesh data in MeshLib to extract the regional mesh data corresponding to different cutting ranges.

4. The visualization method for femoral cutting according to claim 1, wherein The performing cutting rendering in the femur model to be reconstructed based on the pose data and the rendering data of the region corresponding to the pose data includes: Obtaining a pre-generated saw blade model, and determining the spatial position of the saw blade model in the femur model to be reconstructed based on the pose data, and determining the region where the spatial position is located to obtain the region corresponding to the pose data; Generating a rendering layer of the corresponding region based on the rendering data of the corresponding region at the first time node and superimposing it on the femur model to be reconstructed, obtaining the femur model to be reconstructed with the rendering layer of the corresponding region superimposed, and rendering the saw blade model at the corresponding spatial position; Perform a cutting operation on the mesh data of the femur model to be reconstructed based on the spatial position and the mesh data of the saw blade model to obtain the mesh data of the cut femur model. Visualize the cut femur model based on the femur model to be reconstructed with the superimposed corresponding region rendering layer and the mesh data of the cut femur model at a second time node, where the second time node is later than the first time node.

5. The visualization method for femoral cutting according to claim 1, wherein Generating and caching the rendering data for each region based on the mesh data of each region includes: Processing the mesh data of each region according to a preset rendering rule in a preset rendering platform to generate the rendering data of each region, where the rendering effects presented by the rendering data of different regions are different. Cache the rendering data of each region.

6. The visualization method for femoral cutting according to claim 1, wherein, After extracting the virtual cutting plane mesh data from the pre-acquired femur mesh data to be reconstructed based on the cutting plane selection parameter, it further includes: Render a virtual cutting plane in the femur model to be reconstructed based on the virtual cutting plane mesh data.

7. The visualization method for femoral cutting according to claim 1, wherein After performing cutting rendering in the femur model to be reconstructed based on the pose data and the rendering data of the region corresponding to the pose data, it further includes: Visualize the cut femur model obtained by cutting the femur model to be reconstructed and record the mesh data of the cut femur. Obtain the cutting plane replacement parameter input by the user, and extract the reselected virtual cutting plane mesh data from the mesh data of the cut femur based on the cutting plane replacement parameter. Perform a Boolean operation based on the reselected virtual cutting plane mesh data and the mesh data of the cut femur to extract the regional mesh data corresponding to different cutting ranges from the cut femur model. Generate and cache the rendering data for each region of the cut femur model based on the regional mesh data extracted from the cut femur model. Obtain the pose data of the surgical instrument after reselecting the cutting plane, and perform cutting rendering in the cut femur model based on the pose data after reselecting the cutting plane and the rendering data of its corresponding region.

8. A terminal device, characterized in that, The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the computer program is configured to implement the steps of the femur cutting visualization method according to any one of claims 1 to 7.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, it implements the steps of the femur cutting visualization method according to any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the steps of the femur cutting visualization method according to any one of claims 1 to 7.

Images