Surgical path planning methods, systems, devices, media, and surgical operating systems

Abstract

The application discloses a surgical path planning method, system, device, medium and surgical operation system, and the method comprises the steps of constructing a digital twin human body model based on the associated parameters of a target object; acquiring the current contour information of the target object to construct a human body three-dimensional model; registering the digital twin human body model and the human body three-dimensional model to obtain a registration result; determining an initial surgical path planning of a target organ in the digital twin human body model; and converting the initial surgical path planning into a target surgical path planning corresponding to the target organ in the human body three-dimensional model. The application adopts an interventional surgery planning based on a 3D digital twin human body model of a meta universe, effectively improves the accuracy and efficiency of the determination of the interventional surgery path planning, guarantees the safety and reliability of subsequent actual surgery execution, simplifies the whole work flow of the interventional surgery from surgery planning to surgery execution, improves the visual sense and experience of the doctor in surgery planning, and avoids unnecessary errors and radiation hazards.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of surgical control, in particular to an interventional surgery path planning method, system, device, medium and surgery operation system based on meta universe. BACKGROUND

[0002] The general steps of interventional puncture surgery are as follows: image analysis is performed according to the preoperative CT (computed tomography) image of the patient, the tissue and lesion are segmented, and the surgical plan is specified according to the position of the tissue and lesion. The surgical plan includes the needle entry point position and the target point position. Then the doctor performs interventional puncture surgery according to the surgical plan. During the execution of the puncture process, after the puncture needle enters the patient's body for a distance, the doctor will move the patient into the CT aperture for puncture site scanning, and judge whether the position of the puncture needle deviates according to the scanning result; the number of scans required varies according to the doctor's experience and the complexity of the operation. Experienced doctors or simple operations may only need to scan once in the middle; otherwise, multiple scans are required for confirmation. Then the patient is moved out of the CT aperture and the puncture or path adjustment is continued. Finally, the doctor stops the puncture when he believes that the needle tip has reached the target point, and then performs CT scanning for confirmation. If the expected target point position is not reached, the puncture needs to be performed again, and the scanning confirmation is repeated.

[0003] Therefore, the above-mentioned traditional interventional surgery relies on the doctor to plan the surgery based on the image data of the patient, which has the problems of not being intuitive enough visually, being prone to cause the planned path to pass through important organs or tissues (such as blood vessels), and having low planning accuracy and efficiency. In addition, in the execution of the puncture surgery process, the CT and other imaging devices are used to confirm whether the puncture needle is in the planned direction and whether the needle tip of the puncture needle reaches the target position, which increases the risk of radiation hazards to the doctor and the patient. SUMMARY

[0004] The technical problem to be solved by the present application is to overcome the problems of low planning accuracy and efficiency in the path planning of interventional surgery in the prior art, which relies on the doctor to plan the surgery based on the image data of the patient, and the risk of radiation hazards to the doctor and the patient. The present application provides an interventional surgery path planning method, system, device, medium and surgery operation system based on meta universe.

[0005] The present application solves the above technical problems by the following technical solutions:

[0006] The present application provides an interventional surgery path planning method based on meta universe, which comprises:

[0007] constructing a digital twin human body model of the target object based on the correlation parameters of the target object, the digital twin human body model corresponding to an actual state of the target object;

[0008] acquiring current contour information of the target object, and constructing a human body three-dimensional model;

[0009] aligning the digital twin human body model and the human body three-dimensional model to obtain an alignment result;

[0010] determining an initial surgical path plan corresponding to a target organ in the digital twin human body model after alignment based on the correlation parameters;

[0011] converting the initial surgical path plan to a target surgical path plan corresponding to the target organ in the human body three-dimensional model according to the alignment result.

[0012] Preferably, the step of constructing a digital twin human body model of the target object based on the correlation parameters of the target object comprises:

[0013] constructing an initial human body model of the target object based on image data and / or physiological state data of each tissue organ of the target object;

[0014] acquiring action state data of the target object;

[0015] fusing the action state data to the initial human body model to construct the digital twin human body model of the target object;

[0016] and / or,

[0017] The step of acquiring current contour information of the target object and constructing a human body three-dimensional model comprises:

[0018] acquiring the current contour information of the target object using a structured light camera;

[0019] constructing the human body three-dimensional model based on the current contour information.

[0020] Preferably, the step of constructing an initial human body model of the target object based on the correlation parameters comprises:

[0021] constructing the initial human body model of the target object using a volume reconstruction method based on the correlation parameters.

[0022] Preferably, the step of aligning the digital twin human body model and the human body three-dimensional model to obtain an alignment result comprises:

[0023] rigidly transforming the digital twin human body model and the human body three-dimensional model until the digital twin human body model and the human body three-dimensional model are rigidly transformed to the same position and pose;

[0024] performing elastic deformation processing on the human body three-dimensional model;

[0025] performing elastic registration on the digital twin human body model and the human body three-dimensional model after elastic deformation processing to obtain the registration result.

[0026] Preferably, the step of performing elastic deformation processing on the human body three-dimensional model comprises:

[0027] performing elastic deformation processing on the human body three-dimensional model using a radial basis function;

[0028] and / or,

[0029] The step of performing elastic registration on the digital twin human body model and the human body three-dimensional model after elastic deformation processing to obtain the registration result comprises:

[0030] performing elastic registration on the digital twin human body model and the human body three-dimensional model after elastic deformation processing using a non-rigid registration algorithm to obtain the registration result.

[0031] Preferably, after the step of determining the initial surgical path planning corresponding to the target organ in the registered digital twin human body model, and before the step of converting the initial surgical path planning into the target surgical path planning corresponding to the target organ in the human body three-dimensional model according to the registration result, the method further comprises:

[0032] simulating a surgical execution process based on the initial surgical path planning and obtaining a simulation result;

[0033] When the simulation result represents a successful simulation, the step of converting the initial surgical path planning into the target surgical path planning corresponding to the target organ in the human body three-dimensional model according to the registration result is performed;

[0034] When the simulation result represents an unsuccessful simulation, the initial surgical path planning is corrected until the corresponding simulation result represents a successful simulation.

[0035] The present application also provides an intervention surgical path planning system based on the metaverse, which comprises:

[0036] a digital twin model construction module configured to construct a digital twin human body model of a target object based on associated parameters of the target object, the digital twin human body model corresponding to an actual state of the target object;

[0037] a human body three-dimensional model construction module configured to acquire current contour information of the target object and construct a human body three-dimensional model;

[0038] a registration result acquisition module configured to register the digital twin human body model and the human body three-dimensional model and acquire a registration result;

[0039] an initial surgical path planning module configured to determine, based on the association parameter, an initial surgical path planning corresponding to a target organ in the registered digital twin human body model;

[0040] a target surgical path planning module configured to convert the initial surgical path planning into a target surgical path planning corresponding to the target organ in the human body three-dimensional model according to the registration result.

[0041] Preferably, the digital twin model construction module comprises:

[0042] an initial human body model construction unit configured to construct an initial human body model of the target object based on image data and / or physiological state data of each tissue organ of the target object;

[0043] an action state data acquisition unit configured to acquire action state data of the target object;

[0044] a digital twin human body model construction unit configured to fuse the action state data to the initial human body model and construct the digital twin human body model of the target object;

[0045] and / or,

[0046] the human body three-dimensional model construction module comprises:

[0047] a contour information acquisition unit configured to acquire the current contour information of the target object by using a structured light camera;

[0048] a human body three-dimensional model construction unit configured to construct the human body three-dimensional model based on the current contour information.

[0049] Preferably, the initial human body model construction unit is configured to construct the initial human body model of the target object by using a volume reconstruction method based on the association parameter.

[0050] Preferably, the registration result acquisition module comprises:

[0051] a rigid transformation unit configured to perform rigid transformation on the digital twin human body model and the human body three-dimensional model until the digital twin human body model and the human body three-dimensional model are rigidly transformed to the same position and pose;

[0052] an elastic deformation processing unit, configured to perform elastic deformation processing on the human body three-dimensional model;

[0053] a registration result acquisition unit, configured to perform elastic registration on the digital twin human body model and the human body three-dimensional model after elastic deformation processing, and acquire the registration result.

[0054] Preferably, the elastic deformation processing unit is configured to perform elastic deformation processing on the human body three-dimensional model by using a radial basis function;

[0055] and / or,

[0056] The registration result acquisition unit is configured to perform elastic registration on the digital twin human body model and the human body three-dimensional model after elastic deformation processing by using a Non-rigid ICP algorithm (Non-rigid registration algorithm), and acquire the registration result.

[0057] Preferably, the interventional surgery path planning system further comprises:

[0058] a simulation result acquisition module, configured to simulate a surgery execution process based on the initial surgery path planning and acquire a simulation result;

[0059] a judgment module, configured to call the target surgery path planning module when the simulation result indicates simulation success;

[0060] The judgment module is further configured to call a path planning correction module to correct the initial surgery path planning when the simulation result indicates simulation failure, until the simulation result indicates simulation success.

[0061] The present application also provides an interventional surgery operation system, comprising a server, and an image device, a structured light camera, a display terminal and a surgery robot in communication connection with the server;

[0062] The image device is configured to collect image data of each tissue organ of a target object;

[0063] The structured light camera is configured to acquire current contour information of the target object;

[0064] The server comprises the above-mentioned interventional surgery path planning system based on the meta universe, and is configured to output a target surgery path planning corresponding to a target organ in a human body three-dimensional model;

[0065] The display terminal is configured to display the target surgery path planning;

[0066] The surgery robot is configured to perform a corresponding surgery operation after receiving a control instruction generated by the server based on the target surgery path planning.

[0067] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the meta-universe-based interventional surgery path planning method when executing the computer program.

[0068] The application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the meta-universe-based interventional surgery path planning method.

[0069] On the basis of common knowledge in the art, the preferred conditions can be combined arbitrarily, that is, to obtain each preferred embodiment of the application.

[0070] The positive progress effect of the application is that:

[0071] In the application, a digital twin human body model is constructed according to the associated parameters of a patient, a human body three-dimensional model is constructed based on the contour information of the patient, and the two types of 3D human body models are registered, so as to realize the interventional surgery planning based on the 3D digital twin human body model of the meta-universe. Through the dynamic elastic registration of the digital twin model human body and the actual human body three-dimensional model, the actual puncture path is calculated in time, with high precision and high efficiency, the precision and efficiency of the interventional surgery path planning determination are effectively improved, the safety and reliability of the subsequent actual surgery execution are ensured, so as to simplify the whole work flow of the interventional surgery from the surgery planning to the surgery execution, greatly improve the sense and experience of the doctor in the surgery planning, make the surgery planning process more intuitive, safe and efficient, effectively avoid the errors and radiation hazards to the doctor and the patient caused by the manual operation, and make the surgery efficiency and safety much higher than those of the traditional interventional surgery. BRIEF DESCRIPTION OF DRAWINGS

[0072] Figure 1 The figure is a flowchart of the meta-universe-based interventional surgery path planning method of embodiment 1 of the application.

[0073] Figure 2 The figure is a flowchart of the meta-universe-based interventional surgery path planning method of embodiment 2 of the application.

[0074] Figure 3 The figure is a module schematic diagram of the meta-universe-based interventional surgery path planning system of embodiment 3 of the application.

[0075] Figure 4 The figure is a module schematic diagram of the meta-universe-based interventional surgery path planning system of embodiment 4 of the application.

[0076] Figure 5 The figure is a structure schematic diagram of the interventional surgery operation system of embodiment 5 of the application.

[0077] Figure 6 This is a schematic diagram of the structure of the electronic device according to Embodiment 6 of the present invention. Detailed Implementation

[0078] The present invention will be further illustrated by way of embodiments below, but the present invention is not limited to the scope of the embodiments described herein.

[0079] Example 1

[0080] like Figure 1 As shown, the interventional surgical path planning method based on the metaverse in this embodiment includes:

[0081] S101. Based on the associated parameters of the target object, construct a digital twin human body model of the target object;

[0082] Among them, the associated parameters of the target object are data that can characterize various physical indicators of the target object, including but not limited to medical image data, various vital signs and physiological signals.

[0083] The constructed metaverse digital twin human body model corresponds to the actual state of the target object, and can dynamically describe the actual physical state of the target object at any time, displaying the dynamic operation process of each organ in the body with high fidelity.

[0084] S102. Obtain the current contour information of the target object and construct a 3D human body model;

[0085] S103. Register the digital twin human body model and the 3D human body model to obtain the registration results;

[0086] S104. Based on the correlation parameters, determine the initial surgical path planning corresponding to the target organ in the registered digital twin human body model;

[0087] S105. Based on the registration results, the initial surgical path planning is converted into the target surgical path planning corresponding to the target organ in the human three-dimensional model.

[0088] By mapping the initial surgical path plan in the digital twin human body model onto the human body 3D model, the target surgical path plan in the human body 3D model is obtained. This allows for timely and high-precision remote surgery to be performed simultaneously on the target object corresponding to the human body 3D model while the surgery is being performed on the digital twin human body model.

[0089] In this embodiment, a digital twin human body model is constructed according to the associated parameters of the patient, a human body three-dimensional model is constructed based on the contour information of the patient, and then the two categories of 3D human body models are registered, realizing the intervention surgery planning of the 3D digital twin human body model based on the meta universe. Through the dynamic elastic registration of the digital twin model human body and the actual human body three-dimensional model, the actual puncture path is calculated in time, accurately and efficiently, the accuracy and efficiency of the intervention surgery path planning determination are effectively improved, and the safety and reliability of the subsequent actual operation execution are guaranteed.

[0090] Embodiment 2

[0091] As Figure 2 shown, the intervention surgery path planning method based on the meta universe of the present embodiment is a further improvement of embodiment 1, specifically:

[0092] In an implementable scheme, step S101 includes:

[0093] S1011, based on the image data and / or physiological state data of each tissue and organ of the target object, constructing an initial human body model of the target object;

[0094] Wherein, the image data includes but is not limited to CT, MR (magnetic resonance examination), PET (positron emission computed tomography) image;

[0095] The physiological state data includes blood pressure, blood flow, heart rate, respiratory rate and other physiological signals.

[0096] Of course, other dimensional parameters can also be included according to the actual planning scene requirements to construct the initial human body model of the target object, so as to ensure that the model construction is more flexible, efficient and high-precision.

[0097] In an implementable scheme, step S1011 includes

[0098] Based on the associated parameters, the initial human body model of the target object is constructed by using volume reconstruction method.

[0099] Of course, other 3D reconstruction techniques except volume reconstruction method can also be used to realize human body 3D reconstruction, as long as the corresponding initial human body model can be constructed based on the associated parameters.

[0100] Specifically, taking CT image as an example, the CT image is obtained by CT scanning the whole body or a part of the target object, and the skin, skeleton, organ, lesion and blood vessel are segmented; by using volume reconstruction technology, combining the CT image and the physiological state data of the current target object, the CT value of all voxels is set to different transparency, and at the same time, the virtual lighting effect is used to generate and display three-dimensional image with different gray scale, that is, the initial human body model of the target object.

[0101] S1012, acquire the action state data of the target object;

[0102] S1013, fuse the action state data to the initial human body model to construct the digital twin human body model of the target object.

[0103] Specifically, the optical action capture technology based on Marker points (reflective marker points) is adopted to capture the operation data of the surgical site range of the current target object, and all the corresponding action state data are obtained, such as standing, squatting, waving, etc.

[0104] Based on the action state data, the corresponding pose parameters are calculated and introduced into the initial human body model to create the digital twin human body model based on the actual operation state of the body of the target object, to dynamically describe the actual body state of the target object at any time, and to display the actual operation state of each organ through the digital twin human body model with high fidelity. In addition, the digital twin human body model is reconstructed in real time based on the above-mentioned method to ensure high-precision representation of the target object.

[0105] In an implementable scheme, step S102 includes:

[0106] S1021, acquire the current contour information of the target object by using a structured light camera;

[0107] S1022, construct a three-dimensional human body model based on the current contour information.

[0108] In an implementable scheme, step S103 includes:

[0109] S1031, perform rigid transformation on the digital twin human body model and the three-dimensional human body model until the digital twin human body model and the three-dimensional human body model are rigidly transformed to the same position and pose;

[0110] S1032, perform elastic deformation processing on the three-dimensional human body model;

[0111] S1033, perform elastic registration on the digital twin human body model and the three-dimensional human body model after elastic deformation processing to obtain a registration result.

[0112] In an implementable scheme, step S1032 includes:

[0113] Perform elastic deformation processing on the three-dimensional human body model by using a radial basis function;

[0114] In an implementable scheme, step S1033 includes:

[0115] The non-rigid registration algorithm is used to perform elastic registration on the digital twin human body model and the three-dimensional human body model after elastic deformation processing, to obtain a registration result.

[0116] In an implementable solution, step S104 includes:

[0117] Based on the image data of each tissue organ, a three-dimensional point cloud-based binary tree searching method is used to determine the needle entry point position;

[0118] Based on the body correlation parameters, an initial lesion region is determined;

[0119] A preset processing rule is used to process the initial lesion region to obtain a target lesion region;

[0120] A target target point position corresponding to the target lesion region is determined;

[0121] Based on the needle entry point position and the target target point position, an initial surgical path planning corresponding to the target organ in the corresponding digital twin human body model is generated.

[0122] Specifically, the process of the automatic path planning method is as follows:

[0123] I. Path target point selection

[0124] (1) Obtain CT images by performing CT scanning on the whole body or a part of the target object, and segment the skin, bones, organs, lesions, blood vessels, and other tissues or organs to obtain lesion masks (masks);

[0125] (2) Connected component analysis: the organ lesion position may not be only one, which is separated into single lesions, and for each lesion, an appropriate target point needs to be extracted for puncture guidance;

[0126] (3) Extract the ROI region (region of interest) of a single lesion for inflation;

[0127] (4) Resample the ROI region, and the resampling scale is the minimum value of the X, Y, and Z spacing acquisition distances (spatial acquisition distances) in the original image scale; Specifically, X, Y, and Z are three directions of the patient coordinate system, and the spatial acquisition distances in each direction are different, and the minimum spatial acquisition distance in the three directions is selected as the resampling scale;

[0128] (5) Update the lesion mask in the ROI region: When there are multiple unconnected lesions, performing connected component analysis will set different label values for the lesions, and multiple label values will interfere with the distance field calculation. Therefore, the last traversed lesion can be determined as the final retained lesion by traversing the ROI region, or other selection rules such as selecting the largest lesion region can be used to determine the final retained lesion. The specific selection method can be determined or adjusted according to the actual situation.

[0129] (6) Hole filling of the lesion mask: The distance field calculation process is similar to the erosion process and is greatly affected by the intermediate holes, which need to be treated as solid. If hole filling is used when extracting the lesion (or target region), hole filling operation is not needed when calculating the target point. If hole filling is not performed when extracting the lesion, hole filling operation needs to be performed when calculating the target point. The specific time of hole filling can be determined or adjusted according to the actual situation.

[0130] (7) Distance field calculation: Calculate the minimum distance value of each voxel in the lesion mask to the boundary to obtain a plurality of minimum distance values of different distances;

[0131] (8) Determine the best target point on the lesion: Based on the plurality of minimum distance values of different distances obtained in step (7), select the maximum minimum distance value as the distance field maximum value. The calculated distance field maximum value may have multiple values. When the number of voxels of the distance field maximum value is 2, the average value of the 5*5*5 cube adjacent to the voxel is calculated, and the point with the maximum average value is selected as the best target point.

[0132] When the number of voxels of the distance field maximum value is greater than 2, the adaptability of special lesion models (tubular, columnar, etc.) is considered. The multiple distance field maximum value voxels P {P1, P2, P3, …} are traversed, and any voxel P0 in the voxel set P corresponding to the lesion is selected. The sum of the distances between the voxel P0 and each distance field maximum value voxel in P is calculated, and the voxel P0 with the minimum sum of distances is obtained by traversing the solution. The point is the best target point, which makes the target point position as close as possible to the center of the lesion and meets the clinical requirements.

[0133] II. Needle entry point selection

[0134] (1) After confirming the target point, a binary tree search method based on three-dimensional point cloud is used to select the needle entry point.

[0135] (2) A path is determined according to the needle entry point and the target point, and whether the path interferes with the non-penetrable region is calculated. If there is interference, the needle entry point is reselected until a non-interfering path is generated.

[0136] Of course, the initial surgical path planning corresponding to the target organ in the digital twin human body model can also be determined according to the doctor's manual intervention. Specifically, whether to use the above automatic planning path, or to determine based on the doctor's manual planning, or to combine automatic planning and manual planning, can be determined or adjusted in a timely manner according to the actual situation.

[0137] In an implementable solution, after step S104 and before step S105, the method further comprises:

[0138] S10501, simulating the surgical execution process based on the initial surgical path planning and obtaining a simulation result;

[0139] S10502, when the simulation result represents a successful simulation, then performing step S105;

[0140] S10503, when the simulation result represents an unsuccessful simulation, then revising the initial surgical path planning until the corresponding simulation result represents a successful simulation, and continuing to perform step S105.

[0141] For the initial surgical path planning obtained above, the surgery is simulated in the digital twin human body model. If the surgical operation cannot be successfully performed (such as if other important tissues are damaged), the current initial surgical path planning is fed back in a timely manner as not meeting the requirements and needs to be revised until the initial surgical path planning can successfully perform the corresponding surgical operation in the patient's body, avoiding the situation that the subsequent surgery execution is unsafe due to unreasonable path planning, and effectively ensuring the accuracy and rationality of the final surgical path planning, and the safety and reliability of the surgery execution.

[0142] The application scenarios to which the interventional surgical path planning scheme of the embodiment is applicable include, but are not limited to, chest and abdominal interventional surgeries, and the surgical procedures include puncture biopsy, gold marker implantation, guide wire implantation, particle implantation, catheter drainage, and ablation therapy.

[0143] In the embodiment, the digital twin human body model is constructed according to the associated parameters of the patient, the human body three-dimensional model is constructed based on the contour information of the patient, and then the two categories of 3D human body models are registered, realizing the interventional surgery planning based on the 3D digital twin human body model based on the metaverse. Through the dynamic elastic registration of the digital twin model human body and the actual human body three-dimensional model, the actual puncture path is calculated in a timely, accurate and efficient manner, effectively improving the accuracy and efficiency of the interventional surgical path planning determination, and ensuring the safety and reliability of the subsequent actual surgery execution.

[0144] Embodiment 3

[0145] As shown in Figure 3 The interventional surgical path planning system based on the metaverse of the embodiment includes:

[0146] The digital twin model construction module 1 is configured to construct a digital twin human body model of the target object based on associated parameters of the target object.

[0147] The associated parameters of the target object are data capable of representing various physical indicators of the target object, including but not limited to medical image data, various vital signs, and physiological signals.

[0148] The constructed digital twin human body model of the metaverse corresponds to the actual state of the target object, can dynamically describe the actual physical state of the target object at any time, and can display the dynamic operation process of each organ in the body with high fidelity.

[0149] The human body three-dimensional model construction module 2 is configured to obtain current contour information of the target object and construct a human body three-dimensional model.

[0150] The registration result acquisition module 3 is configured to register the digital twin human body model and the human body three-dimensional model and acquire a registration result.

[0151] The initial surgery path planning module 4 is configured to determine an initial surgery path planning corresponding to a target organ in the registered digital twin human body model based on the associated parameters.

[0152] The target surgery path planning module 5 is configured to convert the initial surgery path planning to a target surgery path planning corresponding to the target organ in the human body three-dimensional model according to the registration result.

[0153] It should be noted that the implementation principle of the interventional surgery path planning system of the present embodiment is the same as that of the interventional surgery path planning method of embodiment 1, and therefore will not be described here.

[0154] In the present embodiment, a digital twin human body model is constructed based on the associated parameters of the patient, a human body three-dimensional model is constructed based on the contour information of the patient, and then the two types of 3D human body models are registered to implement interventional surgery planning using a 3D digital twin human body model based on the metaverse. Through dynamic elastic registration of the digital twin model human body and the actual human body three-dimensional model, the actual puncture path is calculated in a timely, accurate and efficient manner, effectively improving the accuracy and efficiency of interventional surgery path planning determination and ensuring the safety and reliability of subsequent actual surgery execution.

[0155] Embodiment 4

[0156] As shown in Figure 4 The interventional surgery path planning system based on the metaverse of the present embodiment is a further improvement of embodiment 4, specifically:

[0157] In an implementable scheme, the digital twin model construction module 1 includes:

[0158] The initial human model construction unit 6 is used to construct an initial human model of the target object based on the image data and / or physiological state data of various tissues and organs of the target object;

[0159] The imaging data includes, but is not limited to, CT, MR, and PET images;

[0160] Physiological data include various physiological signals such as blood pressure, blood flow, heart rate, and respiratory rate.

[0161] Specifically, the initial human body model construction unit constructs the initial human body model of the target object based on the associated parameters and using the volume reconstruction method.

[0162] Action state data acquisition unit 7 is used to acquire the action state data of the target object;

[0163] The digital twin human body model building unit 8 is used to fuse motion state data into the initial human body model to build a digital twin human body model of the target object.

[0164] In one feasible solution, the human body 3D model construction module 2 includes:

[0165] Contour information acquisition unit 9 is used to acquire the current contour information of the target object using a structured light camera;

[0166] Human body 3D model construction unit 10 is used to construct a human body 3D model based on the current contour information.

[0167] In one feasible solution, the registration result acquisition module 3 includes:

[0168] The rigid transformation unit 11 is used to perform rigid transformation on the digital twin human body model and the three-dimensional human body model until the digital twin human body model and the three-dimensional human body model are rigidly transformed to the same position and posture.

[0169] Elastic deformation processing unit 12 is used to perform elastic deformation processing on the three-dimensional human body model;

[0170] The registration result acquisition unit 13 is used to perform elastic registration between the digital twin human body model and the elastically deformed 3D human body model to obtain the registration result.

[0171] In one feasible embodiment, the elastic deformation processing unit 12 is used to perform elastic deformation processing on the three-dimensional human body model using radial basis functions;

[0172] In one feasible scheme, the registration result acquisition unit 13 is used to perform elastic registration of the digital twin human body model and the elastically deformed 3D human body model using a non-rigid registration algorithm to obtain the registration result.

[0173] In an implementable solution, the initial operation path planning module 4 comprises:

[0174] A needle entry point position determination unit 14 is configured to determine the needle entry point position based on the image data of each tissue organ by using a three-dimensional point cloud-based binary tree search method.

[0175] An initial lesion region determination unit 15 is configured to determine the initial lesion region based on the body correlation parameters.

[0176] A target lesion region acquisition unit 16 is configured to process the initial lesion region by using a preset processing rule to acquire the target lesion region.

[0177] A target target point position determination unit 17 is configured to determine the target target point position corresponding to the target lesion region.

[0178] An initial operation path planning generation unit 18 is configured to generate the initial operation path planning corresponding to the target organ in the digital twin human body model based on the needle entry point position and the target target point position.

[0179] In an implementable solution, the interventional operation path planning system further comprises:

[0180] A simulation result acquisition module 19 is configured to simulate the operation execution process based on the initial operation path planning and acquire the simulation result.

[0181] A judgment module 20 is configured to, when the simulation result represents a simulation success, call the target operation path planning module 5.

[0182] The judgment module 20 is further configured to, when the simulation result represents a simulation failure, call the path planning correction module 21 to correct the initial operation path planning until the corresponding simulation result represents a simulation success.

[0183] The application scenarios to which the interventional operation path planning solution of this embodiment is applicable include, but are not limited to, chest and abdominal interventional operations, and the operation types include puncture biopsy, gold marker implantation, guide wire implantation, particle implantation, catheter drainage, and ablation therapy.

[0184] It should be noted that the implementation principle of the interventional operation path planning system of this embodiment is the same as that of the interventional operation path planning method of Embodiment 2, and thus will not be described herein again.

[0185] In this embodiment, a digital twin human body model is constructed according to the associated parameters of the patient, a human body three-dimensional model is constructed based on the contour information of the patient, and then the two categories of 3D human body models are registered, realizing the intervention surgery planning using the 3D digital twin human body model based on the meta universe. Through the dynamic elastic registration of the digital twin model human body and the actual human body three-dimensional model, the actual puncture path is calculated in time, with high precision and high efficiency, effectively improving the precision and efficiency of the intervention surgery path planning determination, and guaranteeing the safety and reliability of the subsequent actual operation execution.

[0186] Embodiment 5

[0187] As shown in Figure 5 The intervention surgery operation system of the present embodiment includes but is not limited to a server 21, and an image device 22, a structured light camera 23, a display terminal 24 and a surgery robot 25 connected in communication with the server 21.

[0188] The image device 22 is used to collect image data of each tissue and organ of the target object;

[0189] The image device 22 includes but is not limited to a CT detection device, a nuclear magnetic resonance detection device.

[0190] The structured light camera 23 is used to obtain the current contour information of the target object;

[0191] The server 21 includes the intervention surgery path planning system based on the meta universe in Embodiment 3 or 4, and the server 21 is used to output the target surgery path planning corresponding to the target organ in the human body three-dimensional model;

[0192] The display terminal 24 is used to display the target surgery path planning;

[0193] The display terminal 24 can be a visual display device, or a virtual display terminal.

[0194] In addition, the display terminal 24 can display not only the target surgery path planning, but also the initial surgery path planning, the digital twin model human body, the actual human body three-dimensional model, etc., and the specific display content can be determined or adjusted according to actual needs.

[0195] Specifically, the remote end doctor directly performs the operation on the digital twin human body model through the VR (virtual reality), AR (augmented reality) and the like, combined with the initial operation path planning, and the corresponding operation execution is simulated and displayed on the digital twin human body model, which is convenient for the doctor to operate and observe. At the same time, the operation data, i.e., the initial operation path planning, is converted into the target operation path planning corresponding to the three-dimensional human body model constructed based on the structured light through 3D elastic registration, space transformation and the like, and the target operation path planning is the operation motion parameter of the local robot, so as to realize the remote data synchronous transmission to the local end, the remote synchronous control of the local robot by the remote end doctor, the remote operation on the target object, and the completion of the operation. In addition, the scheme can provide the doctor with the work of operation planning, operation simulation and postoperative evaluation on the digital twin human body before the operation, so as to improve the safety and predictability of the operation.

[0196] The interventional operation system in the embodiment guarantees the whole process of the interventional operation; of course, other functional devices can be integrated and arranged according to actual needs, so as to achieve the effect of flexibly constructing the interventional operation system, and to meet higher requirements of the interventional operation.

[0197] In addition, the target operation path planning obtained based on the above interventional operation path planning scheme realizes the remote synchronous control of the local robot by the remote end doctor, simplifies the whole work flow of the interventional operation from operation planning to operation execution, greatly improves the sense and experience of the doctor in operation planning, makes the operation planning process more intuitive, safe and efficient, automatically executes the operation by the operation robot throughout the process, reduces the artificial time and cost, avoids the error and radiation harm to the doctor and the patient caused by manual operation, and makes the operation efficiency and safety much higher than those of the traditional interventional operation.

[0198] Embodiment 6

[0199] Figure 6 A structural schematic diagram of an electronic device provided in the embodiment 6 of the present application. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method of the above embodiment when executing the program. Figure 6 The electronic device 30 shown is merely an example and should not impose any limitation on the function and use range of the embodiment of the present application.

[0200] As shown in Figure 6 The electronic device 30 can be in the form of a general computing device, for example, it can be a server device. The components of the electronic device 30 can include but are not limited to the above-mentioned at least one processor 31, the above-mentioned at least one memory 32, and a bus 33 connecting different system components including the memory 32 and the processor 31.

[0201] The bus 33 includes an address bus, a data bus, and a control bus.

[0202] The memory 32 can include volatile memory, such as random access memory (RAM) 321 and / or cache memory 322, and can further include non-volatile memory, such as read-only memory (ROM) 323.

[0203] The memory 32 can also include a program / utility 325 having a set (at least one) of program modules 324, including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which

[0204] The processor 31 can execute the various functional applications and data processing of the method of the above embodiments of the present application by running the computer program stored in the memory 32.

[0205] The electronic device 30 can also communicate with one or more external devices 34 such as a keyboard or a pointing device, among others. This communication can occur via Input / Output (I / O) interface 35. Still yet, the model generating device 30 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the public network, such as the Internet, via network adapter 36. As Figure 6 illustrated, network adapter 36 communicates with the other modules of the model generating device 30 via bus 33. It should be appreciated that although not shown, other hardware and / or software modules could be used in conjunction with the model generating device 30. Such as, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0206] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. Indeed, according to an embodiment of the present application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into a plurality of units / modules.

[0207] Embodiment 7

[0208] The present embodiment provides a computer readable storage medium, having stored thereon a computer program, the program being executed by a processor to implement the steps in the method of the above embodiments.

[0209] More specifically, the readable storage media can include, but is not limited to, a portable disc, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0210] In a possible implementation, the application can also be implemented in the form of a program product, which includes program codes for causing the terminal device to execute the steps in the method of realizing the above-mentioned embodiments when the program product is run on the terminal device.

[0211] The program codes for executing the application can be written in any combination of one or more programming languages, and can be executed entirely on the user device, partly on the user device, as a stand-alone software package, partly on the user device and partly on a remote device, or entirely on a remote device.

[0212] Although the above describes specific implementations of the application, those skilled in the art should understand that this is only an example, and the protection scope of the application is defined by the appended claims. Those skilled in the art can make various changes or modifications to these implementations without departing from the principles and essence of the application, and such changes and modifications all fall within the protection scope of the application.

Claims

1. A meta-universe-based interventional surgery path planning method, characterized in that, The interventional surgery path planning method comprises: constructing a digital twin human body model of the target object based on the associated parameters of the target object, the digital twin human body model corresponding to the actual state of the target object; acquiring current contour information of the target object and constructing a human body three-dimensional model; registering the digital twin human body model and the human body three-dimensional model to obtain a registration result; determining an initial surgery path planning corresponding to a target organ in the registered digital twin human body model based on the associated parameters; converting the initial surgery path planning into a target surgery path planning corresponding to the target organ in the human body three-dimensional model according to the registration result. The step of constructing a digital twin human body model of the target object based on the associated parameters of the target object comprises: constructing an initial human body model of the target object based on image data and / or physiological state data of each tissue organ of the target object; acquiring action state data of the target object; fusing the action state data to the initial human body model to construct the digital twin human body model of the target object.

2. The meta-universe-based interventional surgery path planning method of claim 1, wherein The step of acquiring current contour information of the target object and constructing a human body three-dimensional model comprises: using a structured light camera to collect the current contour information of the target object; constructing the human body three-dimensional model based on the current contour information.

3. The meta-universe based interventional surgery path planning method of claim 2, wherein, The step of constructing an initial human body model of the target object based on the associated parameters comprises: constructing the initial human body model of the target object based on the associated parameters using a volume reconstruction method.

4. The metaverse-based interventional surgery path planning method of any one of claims 1-3, wherein, The step of registering the digital twin human body model and the human body three-dimensional model to obtain a registration result comprises: performing rigid transformation on the digital twin human body model and the human body three-dimensional model until the digital twin human body model and the human body three-dimensional model are rigidly transformed to the same position and attitude; performing elastic deformation processing on the human body three-dimensional model; performing elastic registration on the digital twin human body model and the human body three-dimensional model after elastic deformation processing to obtain the registration result.

5. The meta-universe based interventional surgery path planning method of claim 4, wherein, The step of performing elastic deformation processing on the human body three-dimensional model comprises: performing elastic deformation processing on the human body three-dimensional model using a radial basis function; and / or The step of performing elastic registration on the digital twin human body model and the human body three-dimensional model after elastic deformation processing to obtain the registration result comprises: performing elastic registration on the digital twin human body model and the human body three-dimensional model after elastic deformation processing using a non-rigid registration algorithm to obtain the registration result. The method further comprises, after the step of determining an initial surgery path planning corresponding to a target organ in the registered digital twin human body model based on the associated parameters and before the step of converting the initial surgery path planning into a target surgery path planning corresponding to the target organ in the human body three-dimensional model according to the registration result:

6. The metaverse-based interventional surgery path planning method of any one of claims 1-3, wherein, simulating a surgery execution process based on the initial surgery path planning and obtaining a simulation result; ​ when the simulation result represents a successful simulation, then performing the step of converting the initial surgical path planning into a target surgical path planning corresponding to the target organ in the human three-dimensional model according to the registration result; when the simulation result represents an unsuccessful simulation, then revising the initial surgical path planning until the simulation result represents a successful simulation.

7. A meta-universe based interventional surgery path planning system, characterized by, The interventional surgical path planning system comprises: a digital twin model construction module configured to construct a digital twin human model of a target object based on associated parameters of the target object, the digital twin human model corresponding to an actual state of the target object; a human three-dimensional model construction module configured to obtain current contour information of the target object and construct a human three-dimensional model; a registration result acquisition module configured to register the digital twin human model and the human three-dimensional model and acquire a registration result; an initial surgical path planning module configured to determine an initial surgical path planning corresponding to a target organ in the registered digital twin human model based on the associated parameters; a target surgical path planning module configured to convert the initial surgical path planning into a target surgical path planning corresponding to the target organ in the human three-dimensional model according to the registration result; The digital twin model construction module comprises: an initial human model construction unit configured to construct an initial human model of the target object based on image data and / or physiological state data of each tissue organ of the target object; an action state data acquisition unit configured to acquire action state data of the target object; a digital twin human model construction unit configured to fuse the action state data to the initial human model and construct the digital twin human model of the target object.

8. An interventional operating system, characterized by The interventional surgical operation system comprises a server, and an image device, a structured light camera, a display terminal, and a surgical robot in communication connection with the server; The image device is configured to collect image data of each tissue organ of a target object; The structured light camera is configured to obtain current contour information of the target object; The server comprises the interventional surgical path planning system based on the metaverse as claimed in claim 7, and is configured to output a target surgical path planning corresponding to a target organ in a human three-dimensional model; The display terminal is configured to display the target surgical path planning; The surgical robot is configured to perform a corresponding surgical operation after receiving a control instruction generated by the server based on the target surgical path planning.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the interventional surgical path planning method based on the metaverse as claimed in any one of claims 1-6.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the interventional surgical path planning method based on the metaverse as claimed in any one of claims 1-6.

Images