Surgical navigation system, surgical navigation method, computer equipment and storage medium

Abstract

The invention relates to a surgical navigation system, a surgical navigation method, computer equipment and a storage medium. The system comprises a preoperative planning unit used for acquiring a three-dimensional model of a target tissue organ and establishing a mapping relation between a preset operation stage and positioning information of the three-dimensional model; the visual navigation unit is used for acquiring image information of target tissues and organs in an operation; and the processor is used for identifying the current operation stage according to the image information of the target tissue and organ, and automatically adjusting the positioning degree of the three-dimensional model according to the mapping relation between the current operation stage and the positioning information of the three-dimensional model. By adopting the method, the accuracy of adjusting the positioning of the model in the operation can be improved, so that the problem of large operation error caused by inaccurate positioning of the target tissue organ and the boundary of the target tissue organ by a user is avoided, and the difficulty and the risk of the operation are reduced.

Description

Technical Field

[0001] The present application relates to the field of medical imaging technology, and particularly to a surgical navigation system, a surgical navigation method, a computer device, a storage medium, and a computer program product. Background Art

[0002] Minimally invasive surgery (MIS) is a relatively common surgical technique at present. During the operation, the surgeon no longer directly operates on the patient's lesion, but performs the operation through a laparoscopic instrument (or endoscope instrument) inserted into the patient's cavity, and projects the video stream captured by the laparoscope onto a display to observe the intraoperative process. Compared with traditional open surgery, it has small surgical incisions, fast recovery, and simple postoperative care, which is the current development trend of endoscopic surgery.

[0003] However, during the traditional laparoscopic surgery process, due to the limited surgical environment perception under the laparoscope, the surgeon cannot directly obtain the global information around the laparoscopic field of view, and can only rely on the surgeon's experience and skills to grasp the location, depth, and resection range of the patient's lesion, resulting in problems such as inaccurate positioning of the lesion and its boundary, and excessive resection of healthy tissue, making the operation more difficult and risky. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a surgical navigation system, a surgical navigation method, a surgical navigation device, a computer device, a storage medium, and a computer program product that can improve the accuracy of surgical navigation positioning.

[0005] In a first aspect, the present application provides a surgical navigation system. The system includes:

[0006] A preoperative planning unit, configured to obtain a three-dimensional model of a target tissue organ and establish a mapping relationship between a preset surgical stage and the positioning information of the three-dimensional model;

[0007] A visual navigation unit, configured to obtain image information of the target tissue organ during the operation;

[0008] A processor, configured to identify the current surgical stage according to the image information of the target tissue organ, and automatically adjust the positioning of the three-dimensional model according to the mapping relationship between the current surgical stage and the positioning information of the three-dimensional model.

[0009] In one embodiment, before identifying the current surgical stage according to the image information of the target tissue organ, the processor is configured to:

[0010] Screen out target image frames that meet preset screening conditions from the image information of the target tissue organ;

[0011] Perform image enhancement processing on the target image frame to obtain an enhanced image of the target image frame, and use the enhanced image as the target image.

[0012] In one embodiment, in terms of screening out the target image frame that meets the preset screening conditions from the image information of the target tissue organ, the processor is configured to:

[0013] Screen out candidate image frames that meet the first condition, the second condition, and the third condition from the image information as the target image frame; wherein,

[0014] The first condition includes that the clarity of the candidate image frame is greater than the preset degree;

[0015] The second condition includes that a preset surgical instrument is included in the image frame of the candidate image;

[0016] The third condition includes that a target tissue area for the target tissue organ is included in the image frame of the candidate image.

[0017] In one embodiment, in terms of identifying the current surgical stage according to the image information of the target tissue organ, the processor is configured to:

[0018] Perform similarity matching between the target image and reference images corresponding to multiple preset surgical stages respectively, determine the corresponding target surgical stage, and use the target surgical stage as the current surgical stage;

[0019] Wherein, the similarity matching includes performing similarity matching on the surgical instrument included in the target image with respect to the instrument contour, and performing similarity matching on the target tissue area included in the target image with respect to the area contour.

[0020] In one embodiment, before automatically adjusting the pose of the three-dimensional model, the processor is configured to:

[0021] Overlay and display the image information and the three-dimensional model in the initial pose on a preset surgical navigation interface;

[0022] Wherein, the surgical navigation interface is an augmented reality interface or a virtual reality interface.

[0023] In one embodiment, the pose information is used to indicate the pose perspective of the three-dimensional model in the surgical navigation interface;

[0024] In terms of automatically adjusting the pose of the three-dimensional model, the processor is configured to:

[0025] Obtain the current pose information having a mapping relationship with the current surgical stage;

[0026] Automatically adjust the positioning perspective of the three-dimensional model in the surgical navigation interface according to the current positioning information;

[0027] Among them, the three-dimensional models in different surgical stages have a mapping relationship with different positioning information.

[0028] In one embodiment, after automatically adjusting the positioning of the three-dimensional model, the processor is configured to:

[0029] Extract surgical guidance information bound to the current surgical stage from the database; the surgical guidance information includes operation guidance information and instrument guidance information regarding surgical treatment;

[0030] In the surgical navigation interface, superimpose and display the operation guidance information and the instrument guidance information to provide navigation assistance for the surgical treatment in the current surgical stage.

[0031] In one embodiment, in terms of establishing the mapping relationship between the preset surgical stages and the positioning information of the three-dimensional model, the preoperative planning unit is configured to:

[0032] Display the three-dimensional model in multiple window pages respectively; each window page is associated with a corresponding surgical stage;

[0033] Respond to the positioning configuration operations performed by the user on the three-dimensional models in each window page respectively to obtain the positioning information of the three-dimensional model in each window page;

[0034] Establish the mapping relationship between the positioning information in each window page and the corresponding associated surgical stage.

[0035] In one embodiment, in terms of obtaining the positioning information of the three-dimensional model in each window page, the preoperative planning unit is configured to:

[0036] Respond to the touch operation of the user on the three-dimensional model in the target window page to obtain a custom positioning perspective for the three-dimensional model, and determine the positioning information of the three-dimensional model in the target window page according to the custom positioning perspective;

[0037] Among them, the touch operation includes at least one of the user's dragging operation, rotating operation, translating operation, and scaling operation on the three-dimensional model.

[0038] In one embodiment, in terms of obtaining the positioning information of the three-dimensional model in each window page, the preoperative planning unit is configured to:

[0039] In response to the user's triggering operation on the positioning configuration control displayed in the target window page, generate a default positioning view for the 3D model, and determine the positioning information of the 3D model in the target window page according to the default positioning view;

[0040] Wherein, the positioning configuration control is used to indicate configuring the positioning view of the 3D model according to the image view of the reference image of the surgical stage corresponding to the target window page with a mapping relationship.

[0041] In one embodiment, before obtaining the 3D model of the target tissue organ, the preoperative planning unit is used for:

[0042] Obtain multi-modal medical images collected for the target tissue organ;

[0043] Perform image segmentation processing on the multi-modal medical images to obtain segmentation images of multiple tissue regions in the target tissue organ;

[0044] Perform model reconstruction processing on the segmentation images of each tissue region respectively to obtain a regional reconstruction model for each tissue region;

[0045] Perform fusion processing on each regional reconstruction model to obtain a 3D model for the target tissue organ.

[0046] In a second aspect, the present application also provides a surgical navigation method. The method includes:

[0047] Obtain a 3D model of the target tissue organ and the image information of the target tissue organ during the operation;

[0048] Establish a mapping relationship between a preset surgical stage and the positioning information of the 3D model;

[0049] Identify the current surgical stage according to the image information of the target tissue organ, and automatically adjust the positioning of the 3D model according to the mapping relationship between the current surgical stage and the positioning information of the 3D model.

[0050] In a third aspect, the present application also provides a surgical navigation device. The device includes:

[0051] An acquisition module, configured to acquire a 3D model of the target tissue organ and the image information of the target tissue organ during the operation;

[0052] A mapping module, configured to establish a mapping relationship between a preset surgical stage and the positioning information of the 3D model;

[0053] An adjustment module, configured to identify the current surgical stage according to the image information of the target tissue or organ, and automatically adjust the positioning of the three-dimensional model according to the mapping relationship between the current surgical stage and the positioning information of the three-dimensional model.

[0054] In a fourth aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0055] Obtain multi-modal medical images collected for the target tissue or organ;

[0056] Perform image segmentation processing on the multi-modal medical images to obtain segmentation images of multiple tissue regions in the target tissue or organ;

[0057] Perform model reconstruction processing on the segmentation images of each tissue region respectively to obtain a regional reconstruction model for each tissue region;

[0058] Fuse the regional reconstruction models to obtain a three-dimensional model for the target tissue or organ.

[0059] In a fifth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0060] Obtain multi-modal medical images collected for the target tissue or organ;

[0061] Perform image segmentation processing on the multi-modal medical images to obtain segmentation images of multiple tissue regions in the target tissue or organ;

[0062] Perform model reconstruction processing on the segmentation images of each tissue region respectively to obtain a regional reconstruction model for each tissue region;

[0063] Fuse the regional reconstruction models to obtain a three-dimensional model for the target tissue or organ.

[0064] In a sixth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0065] Obtain multi-modal medical images collected for the target tissue or organ;

[0066] Perform image segmentation processing on the multi-modal medical images to obtain segmentation images of multiple tissue regions in the target tissue or organ;

[0067] Performing model reconstruction processing on the segmented images of each of the tissue regions respectively to obtain a regional reconstruction model for each of the tissue regions;

[0068] Performing fusion processing on each of the regional reconstruction models to obtain a three-dimensional model for the target tissue organ.

[0069] For the above surgical navigation system, surgical navigation method, surgical navigation device, computer device, storage medium, and computer program product, first, a mapping relationship between a preset surgical stage and the positioning information of the three-dimensional model is established through a preoperative planning unit, then the current surgical stage is identified according to the intraoperative image information about the target tissue organ, and finally, according to the mapping relationship between the current surgical stage and the positioning information of the three-dimensional model, the positioning of the three-dimensional model is automatically adjusted so that the positioning of the three-dimensional model can be automatically adjusted as the surgical stage changes. Thus, compared with the prior art, the surgical navigation process is optimized, the accuracy of adjusting the model positioning during the operation is improved, and furthermore, the problem of large surgical errors caused by inaccurate positioning of the target tissue organ and its boundary by the user is avoided, thereby reducing the difficulty and risk of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0071] Figure 1 It is a structural diagram of a surgical navigation system shown according to an exemplary embodiment.

[0072] Figure 2 It is a structural diagram of another surgical navigation system shown according to an exemplary embodiment.

[0073] Figure 3 It is a flowchart of steps for establishing a mapping relationship between a surgical stage and positioning information shown according to an exemplary embodiment.

[0074] Figure 4 It is a flowchart of a surgical navigation method shown according to an exemplary embodiment.

[0075] Figure 5 It is a flowchart of another surgical navigation method shown according to an exemplary embodiment.

[0076] Figure 6 It is a block diagram of a surgical navigation device shown according to an exemplary embodiment.

[0077] Figure 7 It is a block diagram of a computer device for surgical navigation shown according to an exemplary embodiment.

[0078] Figure 8 is a block diagram of a computer-readable storage medium for surgical navigation shown according to an exemplary embodiment.

[0079] Figure 9 is a block diagram of a computer program product for surgical navigation shown according to an exemplary embodiment. Detailed implementation manners

[0080] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0081] The term "and / or" in the embodiments of the present application refers to any and all possible combinations including one or more of the associated listed items. It should also be noted that when used in this specification, "including / comprising" specifies the presence of the stated features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements and / or components and / or groups thereof.

[0082] The surgical navigation system provided by the embodiments of the present application can be a laparoscopic surgical system or an endoscopic surgical system. Corresponding surgical instruments (including, for example, laparoscopes, endoscopes, cameras, etc.) are inserted into the body cavity through incisions on the patient's body surface, and the situation information in the body cavity is obtained through the surgical instruments, and surgical operations are performed according to the situation information. Among them, the surgical navigation system can be a computer device, and the computer device can be a terminal or a server. Taking the terminal as an example, its internal structure diagram can be as Figure 1 shown. The surgical navigation system includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus.

[0083] In one embodiment, the processor of the surgical navigation system is used to provide computing and control capabilities.

[0084] In one embodiment, the memory of the surgical navigation system includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium.

[0085] In one embodiment, the communication interface of the surgical navigation system is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a carrier network, NFC (Near Field Communication) or other technologies. When the computer program is executed by the processor, a surgical navigation method is implemented.

[0086] In one embodiment, the display screen of the surgical navigation system may be a liquid crystal display screen or an electronic ink display screen. The input device of the surgical navigation system may be a touch layer covering the display screen, or buttons, trackballs or touchpads provided on the housing of the surgical navigation system, or an external keyboard, touchpad or mouse, etc.

[0087] Those skilled in the art can understand that Figure 1 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the surgical navigation system to which the solution of the present application is applied. The specific surgical navigation system may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0088] In an exemplary embodiment, the present application provides another surgical navigation system 1, taking the surgical navigation system applied to Figure 1 as an example for illustration. As Figure 2 shown, Figure 2 is a schematic diagram of a surgical navigation system shown according to an exemplary embodiment. The surgical navigation system 1 includes a preoperative planning unit 2, a visual navigation unit 3 and a processor 4.

[0089] In one embodiment, the preoperative planning unit 2 is configured to obtain a three-dimensional model of a target tissue organ and establish a mapping relationship between a preset surgical stage and the positioning information of the three-dimensional model.

[0090] Wherein, the three-dimensional model is a three-dimensional reconstruction model pre-constructed for the target tissue organ of the target object; and, the preset surgical stage is a surgical step in the surgical plan pre-planned by the surgeon based on the target tissue organ of the target object, and there are at least two surgical steps in the surgical plan.

[0091] In some embodiments, the positioning information is used to indicate the positioning perspective of the three-dimensional model in the surgical navigation interface. That is, the three-dimensional model can be used for display in the surgical navigation interface, and the three-dimensional model can display the corresponding positioning perspective in the surgical navigation interface based on the configured positioning information.

[0092] In some embodiments, the three-dimensional models in different surgical stages have a mapping relationship corresponding to different positioning information.

[0093] Specifically, the preoperative planning unit 2 can plan a positioning information of the three-dimensional model for each surgical stage, and bind each surgical stage to its corresponding positioning information to establish a mapping relationship therebetween.

[0094] As an example, a physician pre-plans a liver tumor resection plan for the liver tissue and organ of a patient object, and the liver tumor resection plan includes a first surgical step, a second surgical step, a third surgical step, and a fourth surgical step. Then, the physician or the pre-operative planning unit 2 can configure a model positioning perspective X1 for the first surgical step, configure a model positioning perspective X2 for the second surgical step, configure a model positioning perspective X3 for the third surgical step, and configure a model positioning perspective X4 for the fourth surgical step; finally, the pre-operative planning unit 2 binds the first surgical step and the model positioning perspective X1, the second surgical step and the model positioning perspective X2, the third surgical step and the model positioning perspective X3, and the fourth surgical step and the model positioning perspective X4 respectively to establish a corresponding mapping relationship.

[0095] In one embodiment, the visual navigation unit 3 is used to obtain image information of the intraoperative target tissue and organ.

[0096] In some embodiments, the image information is an image video stream that is real-time collected for the target tissue and organ during the process of the surgeon performing the corresponding surgical operation. Among them, the image information includes a target image of the intraoperative target tissue and organ.

[0097] In some embodiments, the target image is a key image frame of the corresponding image video stream regarding the intraoperative target tissue and organ. Among them, in the key image frame, a surgical instrument and a target tissue area with corresponding contour sizes and shapes are shown.

[0098] Among them, the target tissue area is a preset area of interest in the target tissue and organ, such as a tumor area; the surgical instrument is a surgical instrument used by the surgeon during the surgical operation, such as a scalpel.

[0099] In one embodiment, the processor 4 is used to identify the current surgical stage according to the image information of the target tissue and organ, and automatically adjust the positioning of the three-dimensional model according to the mapping relationship between the current surgical stage and the positioning information of the three-dimensional model.

[0100] In one embodiment, in terms of identifying the current surgical stage according to the image information of the target tissue and organ, the processor 4 is specifically configured to perform: performing similarity matching on the target image with the reference images corresponding to multiple preset surgical stages respectively, determining the target surgical stage corresponding to the matching, and taking the target surgical stage as the current surgical stage.

[0101] Among them, the similarity matching includes performing similarity matching on the surgical instrument included in the target image regarding the instrument contour, and performing similarity matching on the target tissue area included in the target image regarding the area contour.

[0102] As an example, the processor 4 first obtains the target image of the intraoperative target tissue organ included in the image information, as well as the reference images corresponding to each of the preset multiple surgical stages; then, the processor 4 determines the instrument contour information of the processor instrument and the area contour information of the target tissue area from each image; then, the instrument contour information and area contour information of the target image are respectively subjected to similarity matching with the instrument contour information and area contour information of the reference images corresponding to each surgical stage to determine the target reference image with the highest corresponding similarity degree, and the surgical stage corresponding to the target reference image is used as the current surgical stage.

[0103] Among them, the instrument contour information may include the contour size and shape features of the processor instrument, and the area contour information may include the contour size and shape features of the target tissue area.

[0104] In one embodiment, before automatically adjusting the pose of the three-dimensional model, the processor 4 is specifically configured to perform: superimposing and displaying the image information and the three-dimensional model in the initial pose on a preset surgical navigation interface.

[0105] Among them, the surgical navigation interface may be an augmented reality interface or a virtual reality interface.

[0106] Among them, the Augmented Reality Interface is a technology that combines virtual information (i.e., the three-dimensional model) with the real world (i.e., the intraoperative image information). It uses computer vision and sensor technologies to superimpose virtual images or information on the real world, thereby creating an enhanced reality experience.

[0107] Among them, the Virtual Reality Interface is a virtual environment created through computer technology. Users can enter this virtual environment through specific devices (such as head-mounted displays) and interact with the objects and scenes therein.

[0108] Specifically, when the surgeon starts to perform a surgical operation, the processor 4 will display the image information collected during the surgical operation in real time on a preset surgical navigation interface, and superimpose and display the corresponding three-dimensional model in the initial pose on the image information of the surgical navigation interface in an augmented reality manner or a virtual reality manner. Among them, the initial pose of the three-dimensional model refers to the pose information of the three-dimensional model corresponding to the initial surgical stage.

[0109] In one embodiment, in terms of automatically adjusting the pose of the three-dimensional model, the processor 4 is specifically configured to perform the following steps:

[0110] Step 1: Obtain the current pose information having a mapping relationship with the current surgical stage.

[0111] Step 2: Automatically adjust the positioning perspective of the three-dimensional model in the surgical navigation interface according to the current positioning information.

[0112] Specifically, during the surgical process, the processor 4 continuously obtains the current positioning information that has a mapping relationship with the current surgical stage; then, according to the current positioning information, it automatically and continuously adjusts the positioning perspective of the three-dimensional model displayed in the surgical navigation interface to provide medical navigation assistance for the surgical operation performed in the current stage.

[0113] Among them, this medical navigation assistance is used for the processor 4 to more accurately guide the surgeon to perform the surgical operation based on the three-dimensional model under the adjusted positioning perspective, so as to reduce the manual adjustment of the positioning perspective, thereby improving the efficiency and safety of the surgical operation.

[0114] For the above surgical navigation system, first, the preoperative planning unit establishes a mapping relationship between the preset surgical stage and the positioning information of the three-dimensional model, then identifies the current surgical stage according to the intraoperative image information of the target tissue or organ, and finally automatically adjusts the positioning of the three-dimensional model according to the mapping relationship between the current surgical stage and the positioning information of the three-dimensional model, so that the positioning of the three-dimensional model can be automatically adjusted as the surgical stage changes. Thus, compared with the prior art, the surgical navigation process is optimized, the accuracy of adjusting the model positioning during the operation is improved, and the problem of large surgical errors caused by inaccurate positioning of the user for the target tissue or organ and its boundaries is avoided, thereby reducing the difficulty and risk of the operation.

[0115] Those skilled in the art can understand that in the above system units of the specific implementation manner, the disclosed method can be implemented in a more specific manner. For example, the above-described implementation manner in which the processor automatically adjusts the positioning of the three-dimensional model according to the mapping relationship between the current surgical stage and the positioning information of the three-dimensional model is merely illustrative.

[0116] Exemplarily, the way the preoperative planning unit 2 configures the positioning information of the three-dimensional model; or the way the visual navigation unit 3 obtains the intraoperative image information of the target tissue or organ, etc., is merely a collective way, and there can be other division ways in actual implementation. For example, for the positioning information of the three-dimensional model and the intraoperative image information of the target tissue or organ, they can be combined or integrated into another system, or some features can be ignored or not executed.

[0117] In an embodiment, before obtaining the three-dimensional model of the target tissue or organ, the preoperative planning unit 2 is specifically configured to perform the following steps:

[0118] Step 1: Obtain multi-modal medical images collected for the target tissue or organ.

[0119] In one embodiment, the multimodal medical image includes a medical image of a target tissue or organ of the target object acquired by at least one imaging modality.

[0120] For example, the at least two imaging modalities may include modalities such as Magnetic Resonance (MR) imaging, Computed Tomography Angiography (CTA), Digital Subtraction Angiography (DSA), Computed Tomography Perfusion (CTP), Computed Tomography (CT), etc. for providing high-resolution, three-dimensional reconstructed tissue images. Therefore, the acquired multimodal medical images may include MR images, CTA images, DSA images, CTP images, CT images, etc.

[0121] Step two: Perform image segmentation processing on the multimodal medical image to obtain segmentation images of multiple tissue regions in the target tissue or organ.

[0122] Specifically, the preoperative planning unit 2 first performs image region recognition on the multimodal medical image to identify multiple tissue regions in the multimodal medical image, and then performs image segmentation on each of the multiple tissue regions respectively to obtain segmentation images of multiple tissue regions in the target tissue or organ.

[0123] Among them, the identified tissue regions may be organs and tissues in the target object's body, for example, may include organs and tissue regions such as the liver, hepatic artery, portal vein, inferior vena cava, hepatic vein, biliary tract, lesion, etc.

[0124] Step three: Perform model reconstruction processing on the segmentation images of each tissue region respectively to obtain a regional reconstruction model for each tissue region.

[0125] Among them, the regional reconstruction model is a regional three-dimensional model for each tissue region, for example, a regional three-dimensional model for the portal vein tissue, a regional three-dimensional model for the lesion tumor, and so on.

[0126] Step four: Perform fusion processing on each regional reconstruction model to obtain a three-dimensional model of the target tissue or organ.

[0127] Specifically, the preoperative planning unit 2 fuses the regional reconstruction models corresponding to each tissue region to obtain a three-dimensional model of the entire target tissue or organ.

[0128] In an exemplary embodiment, refer toFigure 3 , Figure 3 This is a schematic flowchart of an embodiment for establishing a mapping relationship between surgical stages and positioning information in this application. Specifically, in terms of establishing a mapping relationship between preset surgical stages and the positioning information of a three-dimensional model, the preoperative planning unit 2 is specifically configured to perform the following steps:

[0129] Step a1: Display the three-dimensional model in multiple window pages respectively.

[0130] Among them, each window page is respectively associated with a corresponding surgical stage, and each window page is an interactive page, that is, the user can generate an interaction with the window page through relevant control operations.

[0131] As an example, a target surgical plan has a total of X surgical steps. The preoperative planning unit 2 first takes each surgical step as an independent surgical stage, then configures an independent window page for each surgical stage, and displays them in the display interface respectively, and then displays the retrieved three-dimensional model of the target tissue and organ in the corresponding X window pages respectively.

[0132] Step a2: Respond to the positioning configuration operations performed by the user on the three-dimensional models in each window page respectively, and obtain the positioning information of the three-dimensional models in each window page.

[0133] In one embodiment, in terms of obtaining the positioning information in the window page, the preoperative planning unit 2 is specifically configured to: respond to the touch operation of the user on the three-dimensional model in the target window page, obtain a custom positioning perspective for the three-dimensional model, and determine the positioning information of the three-dimensional model in the target window page according to the custom positioning perspective.

[0134] Among them, the touch operation includes at least one of the user's drag operation, rotation operation, translation operation, and zoom operation on the three-dimensional model.

[0135] As an example, first, the user performs configuration operations such as dragging, rotating, translating, and zooming on the three-dimensional model in the window page; then after the user's configuration operation is completed, the preoperative planning unit 2 can obtain the custom positioning perspective of the corresponding three-dimensional model in the window page at this time, and use this custom positioning perspective as the positioning information of the three-dimensional model in this window page.

[0136] In another embodiment, in terms of obtaining the positioning information in the window page, the preoperative planning unit 2 is specifically configured to: respond to the trigger operation of the user on the positioning configuration control displayed in the target window page, generate a default positioning perspective for the three-dimensional model, and determine the positioning information of the three-dimensional model in the target window page according to the default positioning perspective.

[0137] As an example, first, the preoperative planning unit 2 responds to the triggering operation of the user on the positioning configuration control displayed in the window page, automatically plans and generates the default positioning perspective for the three-dimensional model, and then uses this default positioning perspective as the positioning information of the three-dimensional model in this window page.

[0138] Among them, the positioning configuration control is used to indicate the image perspective of the reference image of the surgical stage corresponding to the target window page with a mapping relationship, and configure the positioning perspective of the three-dimensional model.

[0139] Specifically, the preoperative planning unit 2 automatically plans and generates the default positioning perspective, including: after the positioning configuration control is triggered by the user, the preoperative planning unit 2 first obtains the reference image of the surgical stage corresponding to the target window page with a mapping relationship; then extracts the image perspective in this reference image, and uses this image perspective as the default positioning perspective of the three-dimensional model in this window page.

[0140] Among them, this reference image is obtained based on the pre-recorded reference image information, that is, before the positioning information of the three-dimensional model, the preoperative planning unit 2 pre-obtains the recorded reference surgical video of the target tissue organ, and extracts the reference images for each surgical stage from this reference surgical video.

[0141] Step a3: Establish a mapping relationship between the positioning information in each window page and the corresponding associated surgical stage.

[0142] Specifically, the preoperative planning unit 2 binds the positioning information in each window page to its corresponding associated surgical stage respectively to establish a mapping relationship between the two.

[0143] In one embodiment, before identifying the current surgical stage according to the image information of the target tissue organ, the processor 4 is specifically used to perform the following steps:

[0144] Step 1: Screen out the target image frames that meet the preset screening conditions from the image information of the target tissue organ.

[0145] Specifically, the processor 4 is used to screen out the candidate image frames that meet the first condition, the second condition and the third condition from the image information as the target image frames.

[0146] Among them, the first condition includes that the clarity of the candidate image frame is greater than the preset degree.

[0147] Among them, the second condition includes that the preset processing instrument is included in the image frame of the candidate image frame.

[0148] Among them, the third condition includes that the target tissue area for the target tissue organ is included in the image frame of the candidate image frame.

[0149] As an example, the image information is a real-time video stream collected for the liver tissue and organ. First, the processor 4 extracts all the video image frames from the real-time video stream as candidate image frames, and then performs image detection on each candidate image frame to determine whether the clarity of each candidate image frame is greater than a preset level, whether the image frame includes a preset surgical instrument, and whether the image frame includes a tissue region of the liver tissue and organ, so as to determine whether each candidate image frame meets the first condition, the second condition, and the third condition. Finally, the candidate image frames that meet the first condition, the second condition, and the third condition are used as target image frames.

[0150] Step 2: Perform image enhancement processing on the target image frame to obtain an enhanced image of the target image frame, and use the enhanced image as the target image.

[0151] Among them, the image enhancement processing on the target image frame may include at least one of the following five methods:

[0152] Method 1: Increase the contrast of the target image, that is, by adjusting the gray level of the target image, making the details in the target image more clearly visible.

[0153] Method 2: Perform filtering processing on the target image, that is, use a filter to smooth or sharpen the target image to reduce noise or enhance the details of the target image.

[0154] Method 3: Perform sharpening processing on the target image, that is, by enhancing the edges and details of the target image, making the target image clearer.

[0155] Method 4: Perform pseudo-color processing on the target image, that is, convert the target image presented in gray scale into a pseudo-color image to highlight the contrast of specific structures or tissues.

[0156] Method 5: Fuse the target images, that is, fuse the target images of different modalities or different time points together to obtain more comprehensive information.

[0157] To more clearly illustrate the surgical navigation system provided by the embodiments of the present disclosure, the following uses a specific embodiment to specifically describe the surgical navigation system. In an exemplary embodiment, as Figure 4 shown, a surgical navigation method is provided. Taking the application of this method to a server as an example for description, this method includes the following steps:

[0158] Step S11: Obtain a three-dimensional model of the target tissue and organ, and image information of the target tissue and organ during the operation.

[0159] In an embodiment, before obtaining the three-dimensional model of the target tissue and organ, the server further includes the steps of the following technical implementation methods:

[0160] Step 1: Obtain multi-modal medical images collected from the target tissue or organ.

[0161] Step 2: Perform image segmentation on the multi-modal medical images to obtain segmentation images of multiple tissue regions in the target tissue or organ.

[0162] Step 3: Perform model reconstruction on the segmentation images of each tissue region respectively to obtain a regional reconstruction model for each tissue region.

[0163] Step 4: Perform fusion processing on the regional reconstruction models to obtain a three-dimensional model of the target tissue or organ.

[0164] Step S12: Establish a mapping relationship between the preset surgical stages and the positioning information of the three-dimensional model.

[0165] In one embodiment, in terms of establishing the mapping relationship between the preset surgical stages and the positioning information of the three-dimensional model, the server further includes the steps of the following technical implementation methods:

[0166] Step 1: Display the three-dimensional model in multiple window pages respectively; each window page is associated with a corresponding surgical stage.

[0167] Step 2: Respond to the positioning configuration operations performed by the user on the three-dimensional model in each window page respectively to obtain the positioning information of the three-dimensional model in each window page.

[0168] Step 3: Establish a mapping relationship between the positioning information in each window page and the corresponding associated surgical stage.

[0169] In one embodiment, in terms of obtaining the positioning information of the three-dimensional model in each window page, the server further includes: responding to the touch operation of the user on the three-dimensional model in the target window page to obtain a custom positioning perspective for the three-dimensional model, and determining the positioning information of the three-dimensional model in the target window page according to the custom positioning perspective.

[0170] Wherein, the touch operation includes at least one of the user's drag operation, rotation operation, translation operation, and zoom operation on the three-dimensional model.

[0171] In one embodiment, in terms of obtaining the positioning information of the three-dimensional model in each window page, the server further includes: responding to the trigger operation of the user on the positioning configuration control displayed in the target window page to generate a default positioning perspective for the three-dimensional model, and determining the positioning information of the three-dimensional model in the target window page according to the default positioning perspective.

[0172] Among them, the positioning configuration control is used to indicate the image perspective of the reference image of the surgical stage corresponding to the target window page, and configure the positioning perspective of the three-dimensional model.

[0173] Step S13: Identify the current surgical stage according to the image information of the target tissue organ.

[0174] In one embodiment, before identifying the current surgical stage according to the image information of the target tissue organ, the server further includes the steps of the following technical implementation:

[0175] Step 1: Screen out target image frames that meet the preset screening conditions from the image information of the target tissue organ.

[0176] Step 2: Perform image enhancement processing on the target image frames to obtain enhanced images of the target image frames, and use the enhanced images as target images.

[0177] In one embodiment, in terms of screening out target image frames that meet the preset screening conditions from the image information of the target tissue organ, the server further includes: screening out candidate image frames that meet the first condition, the second condition, and the third condition from the image information as target image frames.

[0178] Among them, the first condition includes that the clarity of the candidate image frame is greater than the preset degree.

[0179] Among them, the second condition includes that the preset surgical instrument is included in the image frame of the candidate image frame.

[0180] Among them, the third condition includes that the target tissue area for the target tissue organ is included in the image frame of the candidate image frame.

[0181] In one embodiment, in terms of identifying the current surgical stage according to the image information of the target tissue organ, the server further includes: performing similarity matching between the target image and the reference images corresponding to multiple preset surgical stages respectively, determining the corresponding target surgical stage for matching, and using the target surgical stage as the current surgical stage.

[0182] Among them, the similarity matching includes performing similarity matching on the instrument contour of the surgical instrument included in the target image, and performing similarity matching on the region contour of the target tissue area included in the target image.

[0183] Step S14: Automatically adjust the positioning of the three-dimensional model according to the mapping relationship between the current surgical stage and the positioning information of the three-dimensional model.

[0184] In one embodiment, before automatically adjusting the positioning of the three-dimensional model, the server further includes: superimposing and displaying the image information and the three-dimensional model in the initial positioning on a preset surgical navigation interface.

[0185] Among them, the surgical navigation interface is an augmented reality interface or a virtual reality interface.

[0186] In one embodiment, the positioning information is used to indicate the positioning perspective of the three-dimensional model in the surgical navigation interface.

[0187] In one embodiment, in terms of automatically adjusting the positioning of the three-dimensional model, the server further includes the steps of the following technical implementation manners:

[0188] Step 1: Obtain the current positioning information having a mapping relationship with the current surgical stage.

[0189] Step 2: According to the current positioning information, automatically adjust the positioning perspective of the three-dimensional model in the surgical navigation interface.

[0190] Among them, the three-dimensional models in different surgical stages correspond to having a mapping relationship with different positioning information.

[0191] In one embodiment, after automatically adjusting the positioning of the three-dimensional model, the server further includes the steps of the following technical implementation manners:

[0192] Step 1: Extract the surgical guidance information bound to the current surgical stage from the database; the surgical guidance information includes operation guidance information and instrument guidance information regarding surgical treatment.

[0193] Step 2: In the surgical navigation interface, superimpose and display the operation guidance information and the instrument guidance information to perform navigation assistance for the surgical treatment of the current surgical stage.

[0194] To more clearly illustrate the surgical navigation method provided by the embodiments of the present disclosure, the following uses a specific embodiment to specifically describe the surgical navigation method. In one exemplary embodiment, refer to Figure 5 , Figure 5 As shown in the flowchart of a surgical navigation method according to another exemplary embodiment, this surgical navigation method is used in a server and specifically includes the following contents:

[0195] Regarding the information data preparation stage before surgical navigation, it includes the following steps S21 - step S26, specifically as follows:

[0196] Step S21: Perform an imaging examination on the patient to obtain multi-modal images of the patient.

[0197] Among them, the multi-modal images belong to the DICOM data format.

[0198] Step S22: Transmit the multi-modal images to a workstation for modeling processing to obtain a three-dimensional reconstruction model of the patient.

[0199] Specifically, the workstation is used to first perform regional segmentation and image reconstruction on the multimodal images to obtain reconstructed images of various organs and tissues in the patient's body. Then, based on the reconstructed images of each organ and tissue, a three-dimensional reconstruction model for the patient is constructed.

[0200] Among them, the organs and tissues in the patient's body may include organ and tissue regions such as the patient's liver, hepatic artery, portal vein, inferior vena cava, hepatic vein, biliary tract, and lesions.

[0201] Step S23: Perform surgical planning operations on the patient based on the three-dimensional reconstruction model to obtain the planned target surgical plan.

[0202] Among them, the surgical planning operations specifically include the following steps:

[0203] Step 1: Determine the vascular system variation information of the patient according to the three-dimensional reconstruction model of the patient.

[0204] In one embodiment, the server displays the three-dimensional reconstruction model of the patient on the display interface so that the doctor can manually view the morphology, size, position, and blood flow of arteries, veins, capillaries, etc., thereby obtaining whether there are abnormalities or variation information such as deformities, dilations, stenoses, and abnormal connections in the patient's vascular system.

[0205] In another embodiment, the server inputs the three-dimensional reconstruction model of the patient or the reconstructed images of the organs and tissues into the pre-trained anomaly detection model to automatically perform anomaly detection to obtain whether there are abnormalities or variation information such as deformities, dilations, stenoses, and abnormal connections in the patient's vascular system.

[0206] Step 2: Simulate multiple surgical resection plans according to the vascular system variation information to obtain the corresponding residual liver volume information under various surgical resection plans.

[0207] Specifically, the server inputs the three-dimensional reconstruction model of the patient and the vascular system variation information into a preset simulation program to configure and simulate multiple surgical resection plans, and perform simulation processing of the corresponding surgical resection for various surgical resection plans. Finally, the simulation program outputs the residual liver volume information corresponding to various surgical resection plans.

[0208] Among them, the configured surgical resection plans include the simulated tumor resection plane and the tumor resection entry path.

[0209] Among them, the configured various surgical resection plans include, for example, anatomical liver lobe surgical plans, hepatic segment and non-anatomical liver enucleation surgical plans, etc., which are not specifically limited here.

[0210] Step 3: Based on various simulated surgical resection plans and the corresponding remnant liver volume information, determine the recommended resection plane and the recommended resection approach for the tumor.

[0211] Specifically, the server displays various simulated surgical resection plans and their corresponding remnant liver volume information on the display interface for doctors to manually determine the optimal resection plane and the optimal resection approach during the operation.

[0212] Step 4: Based on various surgical resection plans, the remnant liver volume information corresponding to each plan, the recommended resection plane, and the recommended resection approach, plan the target surgical plan.

[0213] Specifically, the server displays various simulated surgical resection plans, the remnant liver volume information corresponding to each plan, the recommended resection plane, and the recommended resection approach in the three-dimensional reconstruction model of the patient for doctors to formulate the final target surgical resection plan.

[0214] Among them, the target surgical plan includes: multiple surgical steps, surgical instruments used in each step, the optimal tumor resection plane and the optimal resection approach during the operation.

[0215] Step S24: Transmit the three-dimensional reconstruction model and the target surgical plan to the surgical navigation system for storage.

[0216] Step S25: In response to the doctor's operation of calling out the three-dimensional reconstruction model of the patient in the surgical navigation system, display the three-dimensional reconstruction model and the corresponding target surgical plan on the interaction interface.

[0217] Specifically, the server configures an independent window for each surgical step of the patient and displays the three-dimensional reconstruction model of the patient and the partial target surgical plan of the corresponding surgical step in each independent window.

[0218] Step S26: Based on the doctor's configuration operations in the interaction interface, obtain the model positioning information and surgical guidance information configured for the three-dimensional reconstruction model.

[0219] Specifically, in each independent window, the server responds to the doctor's configuration operations for the model positioning of the three-dimensional reconstruction model and the editing operations of the surgical guidance information, and obtains the model positioning information and surgical guidance information bound to each surgical step in the target surgical plan.

[0220] Among them, the configuration operations for model positioning include: responding to the default positioning or custom positioning selected by the doctor for the three-dimensional reconstruction model.

[0221] Among them, the default pose refers to configuring the pose direction of the three-dimensional reconstruction model as the default pose direction (such as the transverse-coronal-sagittal pose direction), and the custom pose refers to obtaining the custom pose direction of the three-dimensional reconstruction model in the display interface in response to the doctor's gesture operations on the three-dimensional reconstruction model.

[0222] Among them, the doctor's gesture operations include, for example, single-finger drag and rotation, two-finger drag and translation, two-finger pinch and zoom of the three-dimensional reconstruction model, etc.

[0223] Among them, when the three-dimensional reconstruction model is displayed in the display interface, the display interface renders the three-dimensional reconstruction model with a large frame, and uses the model rendering camera to automatically match the aspect ratio with each rendering area where the model is located, so as to avoid deformation of the three-dimensional reconstruction model rendering.

[0224] Among them, the surgical guidance information includes prompt operation guidelines, instrument usage guidelines, and prediction information.

[0225] For the surgical stages in the surgical navigation process, the following steps S27 - step S32 are included, specifically as follows:

[0226] Step S27: Display the real-time surgical video stream during the surgical process in the augmented reality interface, and superimpose and display the three-dimensional reconstruction model in the initial pose direction in the augmented reality interface.

[0227] Step S28: Extract the key image frames including the target organ tissue area from the real-time surgical video stream.

[0228] Specifically, the extraction process of the key image frames may include: screening out the target image frames from each video frame of the real-time surgical video stream that are similar in size and shape to the manually pre-annotated image annotation contour.

[0229] Step S29: Perform image enhancement processing on the key image frames to obtain an enhanced image for the target organ tissue area.

[0230] Among them, performing image enhancement processing on the key image frames may include:

[0231] Increasing contrast: By adjusting the gray level of the image, the details in the image become more clearly visible.

[0232] Filtering processing: Using a filter to smooth or sharpen the image to reduce noise or enhance the details of the image.

[0233] Sharpening processing: By enhancing the edges and details of the image, the image becomes clearer.

[0234] Pseudo-color processing: Converting the gray-scale image into a pseudo-color image to highlight the contrast of specific structures or tissues.

[0235] Image fusion: Fusing images from different modalities or at different time points to obtain more comprehensive information.

[0236] Step S30: Use the pre-trained neural network model to perform image recognition on the enhanced image to obtain the recognition result of the current surgical step.

[0237] Among them, the training process of the neural network model can include a dataset preparation stage and a model training stage.

[0238] Among them, the dataset preparation stage is specifically as follows: First, based on the laparoscopic video images used for training, obtain image frames that meet the requirements; then perform image enhancement processing on the image frames that meet the requirements to obtain enhanced image frames; finally, perform manual annotation on the enhanced image frames to obtain annotation information.

[0239] Among them, the meeting the requirements includes: the clarity and resolution of the image frames meet the preset requirements.

[0240] Among them, the annotation information includes the annotation contour of the corresponding image frame and the classification label regarding the surgical step. Among them, the annotation contour includes the contour of the surgical instrument and the contour of the key organizational structure.

[0241] Among them, the model training stage is specifically as follows: Input the enhanced image frames and annotation information into the deep neural network model for model training to obtain the trained deep neural network model.

[0242] Among them, the input sample data (i.e., the enhanced image frames and annotation information) can be divided into a training set and a test set, where the training set is used for the training of the neural network model, and the test set is used for the performance test of the neural network model.

[0243] Among them, in order to improve the stability of the model performance, a long short-term memory network can be selected for model training.

[0244] Among them, in order to increase the generalization of the obtained model, data augmentation can be performed on the sample data during the training process, including performing angle rotation within a certain range, gray level stretching change, and flipping operation on the sample data.

[0245] Step S31: According to the recognition result of the current surgical step, determine the current model positioning information and the current surgical guidance information bound to the current surgical step.

[0246] Step S32: According to the current model positioning information, adjust the model positioning of the three-dimensional reconstruction model displayed in the augmented reality interface, and superimpose and display the current surgical guidance information in the augmented reality interface.

[0247] In this way, by first establishing a mapping relationship between the preset surgical stages and the positioning information of the three-dimensional model before the operation, then identifying the current surgical stage based on the intraoperative image information of the target tissue and organ, and finally automatically adjusting the positioning of the three-dimensional model according to the mapping relationship between the current surgical stage and the positioning information of the three-dimensional model, the positioning of the three-dimensional model can be automatically adjusted as the surgical stage changes. Thus, compared with the prior art, the process of surgical navigation is optimized, the accuracy of adjusting the model positioning during the operation is improved, and the problem of large surgical errors caused by inaccurate positioning of the target tissue and organ and its boundaries by the user is avoided, thereby reducing the difficulty and risk of the operation.

[0248] It should be understood that although Figures 2 - 5 the steps in the flowchart of Figures 2 - 5 are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover,

[0249] It can be understood that the same / similar parts among the various embodiments of the above methods in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments, and for the related parts, refer to the descriptions of other method embodiments.

[0250] Figure 6 is a block diagram of a surgical navigation device provided by an embodiment of the present application. Referring to Figure 6 , the surgical navigation device 10 includes: an acquisition module 11, a mapping module 12, and an adjustment module 13.

[0251] Among them, the acquisition module 11 is used to acquire the three-dimensional model of the target tissue and organ, and acquire the intraoperative image information of the target tissue and organ;

[0252] Among them, the mapping module 12 is used to establish a mapping relationship between the preset surgical stages and the positioning information of the three-dimensional model;

[0253] Among them, the adjustment module 13 is used to identify the current surgical stage based on the image information of the target tissue and organ, and automatically adjust the positioning of the three-dimensional model according to the mapping relationship between the current surgical stage and the positioning information of the three-dimensional model.

[0254] In some embodiments, before obtaining the three-dimensional model of the target tissue organ, the obtaining module 11 is further configured to:

[0255] Obtain multi-modal medical images collected for the target tissue organ;

[0256] Perform image segmentation processing on the multi-modal medical images to obtain segmentation images of multiple tissue regions in the target tissue organ;

[0257] Perform model reconstruction processing on the segmentation images of each tissue region respectively to obtain a regional reconstruction model for each tissue region;

[0258] Fuse the regional reconstruction models to obtain a three-dimensional model of the target tissue organ.

[0259] In some embodiments, in terms of establishing a mapping relationship between a preset surgical stage and the positioning information of the three-dimensional model, the mapping module 12 is further configured to:

[0260] Display the three-dimensional model in multiple window pages respectively; each window page is associated with a corresponding surgical stage;

[0261] Respond to the positioning configuration operations performed by the user on the three-dimensional model in each window page respectively to obtain the positioning information of the three-dimensional model in each window page;

[0262] Establish a mapping relationship between the positioning information in each window page and the corresponding associated surgical stage.

[0263] In some embodiments, in terms of obtaining the positioning information of the three-dimensional model in each window page, the mapping module 12 is further configured to:

[0264] Respond to the touch operation of the user on the three-dimensional model in the target window page to obtain a custom positioning perspective for the three-dimensional model, and determine the positioning information of the three-dimensional model in the target window page according to the custom positioning perspective;

[0265] Wherein, the touch operation includes at least one of a drag operation, a rotation operation, a translation operation, and a zoom operation performed by the user on the three-dimensional model.

[0266] In some embodiments, in terms of obtaining the positioning information of the three-dimensional model in each window page, the mapping module 12 is further configured to:

[0267] In response to the user's triggering operation on the positioning configuration control displayed in the target window page, generate a default positioning perspective for the 3D model, and determine the positioning information of the 3D model in the target window page according to the default positioning perspective;

[0268] Among them, the positioning configuration control is used to indicate that the positioning perspective of the 3D model is configured according to the image perspective of the reference image of the surgical stage corresponding to the target window page with a mapping relationship.

[0269] In some embodiments, before identifying the current surgical stage based on the image information of the target tissue organ, the adjustment module 13 is further configured to:

[0270] Screen out target image frames that meet the preset screening conditions from the image information of the target tissue organ;

[0271] Perform image enhancement processing on the target image frames to obtain enhanced images of the target image frames, and use the enhanced images as target images.

[0272] In some embodiments, in terms of screening out target image frames that meet the preset screening conditions from the image information of the target tissue organ, the adjustment module 13 is further configured to:

[0273] Screen out candidate image frames that meet the first condition, the second condition, and the third condition from the image information as target image frames; among them,

[0274] The first condition includes that the clarity of the candidate image frame is greater than the preset degree;

[0275] The second condition includes that a preset surgical instrument is included in the image frame of the candidate image frame;

[0276] The third condition includes that a target tissue area for the target tissue organ is included in the image frame of the candidate image frame.

[0277] In some embodiments, in terms of identifying the current surgical stage based on the image information of the target tissue organ, the adjustment module 13 is further configured to:

[0278] Perform similarity matching between the target images and the reference images corresponding to multiple preset surgical stages respectively, determine the target surgical stage corresponding to the matching, and use the target surgical stage as the current surgical stage;

[0279] Among them, the similarity matching includes performing similarity matching on the surgical instruments included in the target images regarding the instrument contours, and performing similarity matching on the target tissue areas included in the target images regarding the area contours.

[0280] In some embodiments, before automatically adjusting the pose of the three-dimensional model, the adjustment module 13 is further configured to:

[0281] Superimpose and display the image information and the three-dimensional model in the initial pose on a preset surgical navigation interface;

[0282] Wherein, the surgical navigation interface is an augmented reality interface or a virtual reality interface.

[0283] In some embodiments, the pose information is used to indicate the pose perspective of the three-dimensional model in the surgical navigation interface; in terms of automatically adjusting the pose of the three-dimensional model, the adjustment module 13 is further configured to:

[0284] Obtain current pose information having a mapping relationship with the current surgical stage;

[0285] Automatically adjust the pose perspective of the three-dimensional model in the surgical navigation interface according to the current pose information;

[0286] Wherein, the three-dimensional models in different surgical stages correspond to different pose information having a mapping relationship.

[0287] In some embodiments, after automatically adjusting the pose of the three-dimensional model, the adjustment module 13 is further configured to:

[0288] Extract surgical guidance information bound to the current surgical stage from a database; the surgical guidance information includes operation guidance information and instrument guidance information regarding surgical treatment;

[0289] In the surgical navigation interface, superimpose and display the operation guidance information and the instrument guidance information to provide navigation assistance for the surgical treatment in the current surgical stage.

[0290] Figure 7 It is a block diagram of a computer device 20 provided by an embodiment of the present application. For example, the computer device 20 can be an electronic device, an electronic component, or a server array, etc. Referring to Figure 7 , the computer device 20 includes a processor 21, and the further processor 21 can be a set of processors, which can include one or more processors, and the computer device 20 includes memory resources represented by a memory 22, wherein a computer program, such as an application program, is stored on the memory 22. The computer program stored in the memory 22 can include one or more modules each corresponding to a set of executable instructions. In addition, when the processor 21 is configured to execute the computer program, it implements the surgical navigation method as described above.

[0291] In some embodiments, the computer device 20 is an electronic device, and the computing system in the electronic device can run one or more operating systems, including any of the operating systems discussed above and any commercial server operating system. The computer device 20 can also run any of a variety of additional server applications and / or middleware applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, superservers, database servers, etc. Exemplary database servers include, but are not limited to, database servers commercially available from (International Business Machines) and the like.

[0292] In some embodiments, the processor 21 generally controls the overall operation of the computer device 20, such as operations associated with display, data processing, data communication, and recording operations. The processor 21 can include one or more processor components to execute a computer program to complete all or part of the steps of the above-described method. In addition, the processor components can include one or more modules to facilitate interaction between the processor components and other components. For example, the processor components can include a multimedia module to facilitate controlling the interaction between the user computer device 20 and the processor 21 using the multimedia components.

[0293] In some embodiments, the processor components in the processor 21 can also be referred to as a CPU (Central Processing Unit). The processor components may be an electronic chip with the ability to process signals. The processor can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor component, etc. Additionally, the processor components can be implemented jointly by integrated circuit chips.

[0294] In some embodiments, the memory 22 is configured to store various types of data to support the operation of the computer device 20. Examples of such data include instructions for any application or method operating on the computer device 20, acquisition data, messages, pictures, videos, and the like. The memory 22 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disks, optical disks, or graphene memory.

[0295] In some embodiments, the memory 22 can be a memory module, a TF card, etc., and can store all the information in the computer device 20. The input original data, computer programs, intermediate operation results, and final operation results are all stored in the memory 22. In some embodiments, it stores and retrieves information according to the positions specified by the processor. In some embodiments, with the memory 22, the computer device 20 has a memory function and can ensure normal operation. In some embodiments, the memory 22 of the computer device 20 can be classified into a main memory (RAM) and an auxiliary memory (external memory) according to its use, and there is also a classification method of dividing it into an external memory and an internal memory. The external memory is usually a magnetic medium or an optical disk, etc., which can store information for a long time. The RAM refers to the storage component on the motherboard, which is used to store the data and programs being currently executed, but only temporarily stores the programs and data. When the power is turned off or interrupted, the data will be lost.

[0296] In some embodiments, the computer device 20 may further include: a power supply component 23 configured to perform power management of the computer device 20, a wired or wireless network interface 24 configured to connect the computer device 20 to a network, and an input / output (I / O) interface 25. The computer device 20 can operate based on an operating system stored in the memory 22, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, or the like.

[0297] In some embodiments, the power supply component 23 supplies power to various components of the computer device 20. The power supply component 23 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the computer device 20.

[0298] In some embodiments, the wired or wireless network interface 24 is configured to facilitate communication between the computer device 20 and other devices in a wired or wireless manner. The computer device 20 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G, or 5G), or a combination thereof.

[0299] In some embodiments, the wired or wireless network interface 24 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the wired or wireless network interface 24 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0300] In some embodiments, the input / output (I / O) interface 25 provides an interface between the processor 21 and a peripheral interface module, and the peripheral interface module can be a keyboard, click wheel, buttons, etc. These buttons can include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0301] Figure 8 It is a block diagram of a computer-readable storage medium 30 provided by an embodiment of the present application. A computer program 31 is stored on the computer-readable storage medium 30. When the computer program 31 is executed by a processor, the surgical navigation method as described above is implemented.

[0302] If the units integrated in each functional unit in various embodiments of the present application are implemented in the form of software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium 30. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer-readable storage medium 30 includes several instructions in a computer program 31 to enable a computer device (which can be a personal computer, a system server, or a network device, etc.), an electronic device (such as an MP3, MP4, etc., or can also be a smart terminal such as a mobile phone, a tablet computer, a wearable device, etc., or a desktop computer, etc.) or a processor to execute all or part of the steps of the methods in various embodiments of the present application.

[0303] Figure 9 It is a block diagram of a computer program product 40 provided by an embodiment of the present application. The computer program product 40 includes program instructions 41, and the program instructions 41 can be executed by the processor of the server 20 to implement the surgical navigation method as described above.

[0304] Those skilled in the art should understand that the embodiments of the present application may provide a surgical navigation system, a surgical navigation method, a surgical navigation device 10, a computer device 20, a computer-readable storage medium 30, or a computer program product 40. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product 40 implemented on one or more computer program instructions 41 (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0305] The present application is described with reference to the flowcharts and / or block diagrams of the surgical navigation system, surgical navigation method, surgical navigation device 10, computer device 20, computer-readable storage medium 30, or computer program product 40 in the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by the computer program product 40. These computer program products 40 can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the program instructions 41 executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0306] These computer program products 40 can also be stored in a computer-readable memory that can direct a computer or other programmable data processing devices to work in a specific manner, so that the program instructions 41 stored in the computer program product 40 generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0307] These program instructions 41 can also be loaded onto a computer or other programmable data processing devices, so that a series of operation steps are executed on the computer or other programmable devices to generate a computer-implemented process, and thus the program instructions 41 executed on the computer or other programmable devices provide steps for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0308] It should be noted that the above-mentioned various methods, devices, electronic devices, computer-readable storage media, computer program products, etc. may also include other implementation manners according to the description of the method embodiments. The specific implementation manners may refer to the description of the relevant method embodiments and will not be elaborated herein one by one.

[0309] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

[0310] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A surgical navigation system, characterized in that, including a preoperative planning unit configured to obtain a three-dimensional model of a target tissue or organ and establish a mapping relationship between a preset surgical stage and the positioning information of the three-dimensional model; a visual navigation unit configured to obtain intraoperative image information of the target tissue or organ; a processor configured to identify a current surgical stage based on the image information of the target tissue or organ, and automatically adjust the positioning of the three-dimensional model according to the mapping relationship between the current surgical stage and the positioning information of the three-dimensional model.

2. The system according to claim 1, characterized in that, Before identifying the current surgical stage based on the image information of the target tissue or organ, the processor is configured to: screen out target image frames that meet a preset screening condition from the image information of the target tissue or organ; perform image enhancement processing on the target image frames to obtain enhanced images of the target image frames, and use the enhanced images as target images.

3. The system according to claim 2, characterized in that, In terms of screening out target image frames that meet a preset screening condition from the image information of the target tissue or organ, the processor is configured to: screen out candidate image frames that meet a first condition, a second condition, and a third condition from the image information as target image frames; wherein, the first condition includes that the clarity of the candidate image frame is greater than a preset degree; the second condition includes that a preset surgical instrument is included in the image frame of the candidate image frame; the third condition includes that a target tissue area for the target tissue or organ is included in the image frame of the candidate image frame.

4. The system according to claim 3, characterized in that, In terms of identifying the current surgical stage based on the image information of the target tissue or organ, the processor is configured to: perform similarity matching between the target image and reference images corresponding to a plurality of preset surgical stages respectively, determine a target surgical stage corresponding to the matching, and use the target surgical stage as the current surgical stage; wherein, the similarity matching includes performing similarity matching on the surgical instrument included in the target image with respect to the instrument contour, and performing similarity matching on the target tissue area included in the target image with respect to the area contour.

5. The system according to claim 1, characterized in that, Before automatically adjusting the positioning of the three-dimensional model, the processor is configured to: superimpose and display the image information and the three-dimensional model in an initial position on a preset surgical navigation interface; wherein, the surgical navigation interface is an augmented reality interface or a virtual reality interface.

6. The system according to claim 5, characterized in that, The positioning information is used to indicate the positioning perspective of the three-dimensional model in the surgical navigation interface; In terms of automatically adjusting the positioning of the three-dimensional model, the processor is configured to: obtain current positioning information having a mapping relationship with the current surgical stage; automatically adjust the positioning perspective of the three-dimensional model in the surgical navigation interface according to the current positioning information; wherein, the three-dimensional models in different surgical stages respectively have a mapping relationship with different positioning information.

7. The system according to claim 5, characterized in that, After automatically adjusting the positioning of the three-dimensional model, the processor is configured to: extract surgical guidance information bound to the current surgical stage from a database; the surgical guidance information includes operation guidance information and instrument guidance information regarding surgical treatment; In the surgical navigation interface, the operation guidance information and the instrument guidance information are superimposed and displayed to provide navigation assistance for the surgical treatment in the current surgical stage.

8. The system according to claim 1, characterized in that, In terms of establishing the mapping relationship between the preset surgical stages and the positioning information of the three-dimensional model, the preoperative planning unit is configured to: Display the three-dimensional model in multiple window pages respectively; each of the window pages is associated with a corresponding surgical stage; In response to the user's positioning configuration operations on the three-dimensional model in each of the window pages, obtain the positioning information of the three-dimensional model in each of the window pages; Establish the mapping relationship between the positioning information in each of the window pages and the corresponding associated surgical stage.

9. The system according to claim 8, characterized in that, In terms of obtaining the positioning information of the three-dimensional model in each of the window pages, the preoperative planning unit is configured to: In response to the user's touch operation on the three-dimensional model in the target window page, obtain a custom positioning perspective for the three-dimensional model, and determine the positioning information of the three-dimensional model in the target window page according to the custom positioning perspective; Wherein, the touch operation includes at least one of the user's dragging operation, rotating operation, translating operation, and scaling operation on the three-dimensional model.

10. The system according to claim 8, wherein, In terms of obtaining the positioning information of the three-dimensional model in each of the window pages, the preoperative planning unit is configured to: In response to the user's triggering operation on the positioning configuration control displayed in the target window page, generate a default positioning perspective for the three-dimensional model, and determine the positioning information of the three-dimensional model in the target window page according to the default positioning perspective; Wherein, the positioning configuration control is used to indicate configuring the positioning perspective of the three-dimensional model according to the image perspective of the reference image of the surgical stage corresponding to the target window page with a mapping relationship.

11. The system according to claim 8, wherein, Before obtaining the three-dimensional model of the target tissue organ, the preoperative planning unit is configured to: Obtain multi-modal medical images collected for the target tissue organ; Perform image segmentation processing on the multi-modal medical images to obtain segmentation images of multiple tissue regions in the target tissue organ; Perform model reconstruction processing on the segmentation images of each of the tissue regions respectively to obtain region reconstruction models for each of the tissue regions; Perform fusion processing on each of the region reconstruction models to obtain a three-dimensional model for the target tissue organ.

12. A surgical navigation method, wherein, The method includes: Obtain a three-dimensional model of a target tissue organ and the image information of the target tissue organ during the operation; Establish the mapping relationship between the preset surgical stages and the positioning information of the three-dimensional model; Identify the current surgical stage according to the image information of the target tissue organ, and automatically adjust the positioning of the three-dimensional model according to the mapping relationship between the current surgical stage and the positioning information of the three-dimensional model.

13. A computer device, wherein, Includes: A processor; A memory for storing executable instructions of the processor; Wherein, the processor is configured to execute the executable instructions to implement the surgical navigation method as claimed in claim 12.

14. A computer-readable storage medium, wherein the computer-readable storage medium includes program data, wherein, When the program data is executed by a processor of a computer device, the computer device is enabled to perform the surgical navigation method as described in claim 12.

Images