Medical image processing method, system, device and storage medium for interventional surgery

Abstract

The embodiment of the specification provides an interventional surgery medical image processing method, the method comprises the following steps: acquiring a mode of a planned interventional path; acquiring preoperative enhanced image; segmenting a first target structure set of the preoperative enhanced image to obtain a first segmented image of the first target structure set; acquiring intraoperative scanning image; segmenting a second target structure set of the intraoperative scanning image to obtain a second segmented image of the second target structure set; registering the first segmented image and the second segmented image; determining the spatial position of a third target structure set in the surgery, the elements of the third target structure set are selected based on the mode of the planned interventional path; planning the interventional path based on the spatial position of the third target structure set in the surgery, and performing risk assessment based on the interventional path; wherein at least one element in the third target structure set is included in the first target structure set, and at least one element in the third target structure set is not included in the second target structure set.

Description

Case Analysis

[0001] This application is a divisional application of Chinese application filed on June 30, 2022, with application number 202210764281.2 entitled "Method, System, Apparatus and Storage Medium for Medical Image Processing in Interventional Surgery". Technical Field

[0002] This specification relates to the field of image processing technology, and in particular to a method, system, device, and computer storage medium for medical image processing in interventional surgery. Background Technology

[0003] Preoperative planning for interventional procedures is an important part of assisting interventional procedures. The accuracy of the planning directly affects the accuracy of the interventional path during the interventional procedure, thus affecting the outcome of the interventional procedure. On the other hand, actual interventional procedures are often more complex, with some variable factors, which also affect whether the interventional procedure can be performed well.

[0004] Previous techniques not only lacked sufficient precision in preoperative planning, but also, due to inaccurate planning, resulted in a simplistic workflow, poor risk avoidance, and consequently, poor surgical outcomes, significantly limiting the scope for workflow optimization. Therefore, the need arises to provide a more comprehensive and optimized image workflow for assisted interventional surgery to substantially improve surgical efficiency. Summary of the Invention

[0005] One embodiment of this specification provides a method for medical image processing in interventional surgery. The method includes: acquiring a pattern for planning an interventional path; acquiring preoperative enhanced images; segmenting a first target structure set of the preoperative enhanced images to obtain a first segmented image of the first target structure set; acquiring intraoperative scan images; segmenting a second target structure set of the intraoperative scan images to obtain a second segmented image of the second target structure set; wherein the first target structure set and the second target structure set have an intersection; registering the first segmented image and the second segmented image; determining the spatial location of a third target structure set during surgery, wherein the elements of the third target structure set are selected based on the pattern for planning the interventional path; planning an interventional path based on the spatial location of the third target structure set during surgery; and performing a risk assessment based on the interventional path; wherein at least one element in the third target structure set is included in the first target structure set, and at least one element in the third target structure set is not included in the second target structure set.

[0006] One embodiment of this specification provides a medical image processing system for interventional surgery, comprising: a first acquisition module for acquiring a pattern for planning an interventional path; a second acquisition module for acquiring preoperative enhanced images; a first segmentation module for segmenting a first target structure set of the preoperative enhanced images to obtain a first segmented image of the first target structure set; a third acquisition module for acquiring intraoperative scan images; a second segmentation module for segmenting a second target structure set of the intraoperative scan images to obtain a second segmented image of the second target structure set; a deformation registration module for registering the first segmented image and the second segmented image to determine the spatial location of a third target structure set during surgery, wherein the elements of the third target structure set are selected based on the pattern of the planned interventional path; and a risk assessment module for planning an interventional path based on the spatial location of the third target structure set during surgery and performing a risk assessment based on the interventional path; wherein at least one element of the third target structure set is included in the first target structure set, and at least one element of the third target structure set is not included in the second target structure set.

[0007] One embodiment of this specification provides a medical image processing apparatus for interventional surgery, including a processor, the processor being configured to execute the medical image processing method for interventional surgery described in any embodiment.

[0008] One embodiment of this specification provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the medical image processing method for interventional surgery as described in any embodiment. Attached Figure Description

[0009] This specification will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein: Figure 1 These are schematic diagrams illustrating application scenarios of a medical image processing system for interventional surgery, based on some embodiments of this specification. Figure 2 This is an exemplary flowchart of a method for processing medical images for interventional surgery, provided according to some embodiments of this specification; Figure 3 An exemplary flowchart of the segmentation process involved in the interventional surgical medical image processing method provided according to some embodiments of this specification; Figure 4 This is an exemplary flowchart illustrating the process of determining the positioning information of an element mask according to some embodiments of this specification; Figure 5This is an exemplary flowchart of a soft connected domain analysis process using element masks as shown in some embodiments of this specification; Figure 6 These are exemplary comparison diagrams showing the effects of coarse segmentation of element masks for soft connected component analysis according to some embodiments of this specification; Figure 7 This is an exemplary flowchart illustrating the process of precisely segmenting elements according to some embodiments of this specification; Figure 8 This is an exemplary schematic diagram showing the determination of the positioning information of the element mask according to some embodiments of this specification; Figure 9 This is an exemplary schematic diagram showing the determination of the positioning information of the element mask according to some embodiments of this specification; Figure 10A This is an exemplary schematic diagram illustrating the determination of sliding direction based on the positioning information of an element mask according to some embodiments of this specification; Figures 10B-10E This is an exemplary schematic diagram illustrating precise segmentation after sliding a window, as shown in some embodiments of this specification; Figure 11 These are exemplary comparison diagrams of segmentation results shown in some embodiments of this specification; Figure 12 This is an exemplary flowchart illustrating the registration process between a first segmented image and a second segmented image as shown in some embodiments of this specification; Figure 13 This is an exemplary flowchart illustrating the process of determining the registration deformation field in some embodiments of this specification; Figure 14 This is an exemplary flowchart illustrating the process of determining the registration deformation field in some embodiments of this specification; Figure 15 These are exemplary demonstration diagrams showing the first segmented image and the second segmented image obtained after segmentation, as illustrated in some embodiments of this specification. Figure 16 This is an exemplary flowchart of determining the intervention risk value of at least some elements in a third target structure set under the rapid planning mode shown in some embodiments of this specification; Figure 17 This is an exemplary flowchart illustrating the determination of intervention risk values ​​for at least some elements in a third target structure set under the precise planning mode shown in some embodiments of this specification; Figure 18 This is an exemplary flowchart of the image anomaly detection process shown in some embodiments of this specification; Figure 19 This is an exemplary flowchart of the postoperative assessment process shown in some embodiments of this specification; Figure 20This is an exemplary flowchart of the postoperative assessment process shown in some embodiments of this specification; Figure 21 This is an exemplary structural block diagram of a medical image processing system 900 for interventional surgery, as shown in some embodiments of this specification. Detailed Implementation

[0010] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some examples or embodiments of this specification. For those skilled in the art, these drawings can be applied to other similar scenarios without creative effort. Unless obvious from the context or otherwise specified, the same reference numerals in the drawings represent the same structures or operations.

[0011] It should be understood that the terms “system,” “device,” “unit,” and / or “module” used herein are one way to distinguish different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0012] As indicated in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0013] Flowcharts are used in this specification to illustrate the operations performed by the system according to embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0014] Figure 1 This is a schematic diagram illustrating an application scenario of the interventional surgical medical image processing system 100 according to some embodiments of this specification.

[0015] In some embodiments, the medical image processing system 100 can be applied to a variety of interventional procedures / treatments. In some embodiments, interventional procedures / treatments may include cardiovascular interventional procedures, oncology interventional procedures, obstetric and gynecological interventional procedures, musculoskeletal interventional procedures, or any other feasible interventional procedures, such as neurointerventional procedures. In some embodiments, interventional procedures / treatments may include percutaneous biopsy, coronary angiography, thrombolysis, stent placement, or any other feasible interventional procedure, such as ablation.

[0016] Medical image processing system 100 may include medical scanning equipment 110, network 120, one or more terminals 130, processing equipment 140, and storage device 150. The connections between the components in medical image processing system 100 can be variable. For example... Figure 1 As shown, medical scanning device 110 can be connected to processing device 140 via network 120. Alternatively, medical scanning device 110 can be directly connected to processing device 140, as indicated by the dashed double-headed arrow connecting medical scanning device 110 and processing device 140. As another example, storage device 150 can be connected to processing device 140 directly or via network 120. As an example, terminal 130 can be directly connected to processing device 140 (as indicated by the dashed arrow connecting terminal 130 and processing device 140) or connected to processing device 140 via network 120.

[0017] Medical scanning device 110 can be configured to scan a subject using high-energy rays (such as X-rays, gamma rays, etc.) to collect scan data related to the subject. The scan data can be used to generate one or more images of the subject. In some embodiments, medical scanning device 110 may include an ultrasound imaging (US) device, a computed tomography (CT) scanner, a digital radiography (DR) scanner (e.g., moving digital radiography), a digital subtraction angiography (DSA) scanner, a dynamic spatial reconstruction (DSR) scanner, an X-ray microscopy scanner, a multimodal scanner, etc., or combinations thereof. In some embodiments, a multimodal scanner may include a computed tomography-positron emission tomography (CT-PET) scanner, a computed tomography-magnetic resonance imaging (CT-MRI) scanner, etc. The subject can be biological or non-biological. By way of example only, the subject may include a patient, an artificial object (e.g., a phantom), etc. As another example, the subject may include a specific site, organ, and / or tissue of a patient.

[0018] like Figure 1As shown, the medical scanning device 110 may include a rack 111, a detector 112, a detection area 113, a worktable 114, and a radiation source 115. The rack 111 can support the detector 112 and the radiation source 115. A scanned object can be placed on the worktable 114 for scanning. The radiation source 115 can emit radiation towards the scanned object. The detector 112 can detect the radiation emitted from the radiation source 115 (e.g., X-rays). In some embodiments, the detector 112 may include one or more detector units. Detector units may include scintillation detectors (e.g., cesium iodide detectors), gas detectors, etc. Detector units may include single-row detectors and / or multi-row detectors.

[0019] Network 120 may include any suitable network that can facilitate the exchange of information and / or data between the medical image processing system 100. In some embodiments, components of one or more medical image processing systems 100 (e.g., medical scanning device 110, terminal 130, processing device 140, storage device 150) may exchange information and / or data with each other via network 120. For example, processing device 140 may obtain image data from medical scanning device 110 via network 120. As another example, processing device 140 may obtain user instructions from terminal 130 via network 120.

[0020] Network 120 may be and / or include public networks (e.g., the Internet), private networks (e.g., local area networks (LANs), wide area networks (WANs), etc.), wired networks (e.g., Ethernet networks, wireless networks (e.g., 802.11 networks, Wi-Fi networks, etc.), cellular networks (e.g., LTE networks), Frame Relay networks, virtual private networks (“VPNs”), satellite networks, telephone networks, routers, hubs, switches, server computers, and / or any combination thereof. By way of example only, network 120 may include cable networks, wired networks, fiber optic networks, telecommunications networks, intranets, wireless local area networks (WLANs), metropolitan area networks (MANs), public switched telephone networks (PSTNs), Bluetooth, etc. TM Network, ZigBee TM A network, a near field communication (NFC) network, or any combination thereof. In some embodiments, network 120 may include one or more network access points. For example, network 120 may include wired and / or wireless network access points such as base stations and / or internet exchange points, through which one or more components of the medical image processing system 100 can connect to network 120 to exchange data and / or information.

[0021] Terminal 130 may include mobile device 131, tablet computer 132, laptop computer 133, etc., or any combination thereof. In some embodiments, mobile device 131 may include smart home devices, wearable devices, mobile devices, virtual reality devices, augmented reality devices, etc., or any combination thereof. In some embodiments, smart home devices may include smart lighting devices, smart electrical device control devices, smart monitoring devices, smart TVs, smart cameras, walkie-talkies, etc., or any combination thereof. In some embodiments, mobile devices may include mobile phones, personal digital assistants (PDAs), gaming devices, navigation devices, point-of-sale (POS) devices, laptops, tablets, desktop computers, etc., or any combination thereof. In some embodiments, virtual reality devices and / or augmented reality devices include virtual reality headsets, virtual reality glasses, virtual reality goggles, augmented reality headsets, augmented reality glasses, augmented reality goggles, etc., or any combination thereof. For example, virtual reality devices and / or augmented reality devices may include Google Glass™, Oculus Rift™, HoloLens™, Gear VR™, etc. In some embodiments, terminal 130 may be part of processing device 140.

[0022] Processing device 140 can process data and / or information obtained from medical scanning device 110, terminal 130, and / or storage device 150. For example, processing device 140 can acquire data acquired by medical scanning device 110 and use this data to generate medical images (such as preoperative enhanced images, intraoperative scan images), and segment the medical images to generate segmentation result data (such as first segmented image, spatial location of blood vessels and lesions during surgery, registration map, etc.). As another example, processing device 140 can acquire medical images, planning mode data (such as precise planning mode data, rapid planning mode data), and / or scanning protocols from terminal 130. As yet another example, processing device 140 can acquire data acquired by medical scanning device 110 (such as segmentation and registration results, interventional risk values, preset weights, weighted risk values, cumulative risk values, image anomaly types, image anomaly degrees, etc.), and use this data to generate interventional paths and / or prompt information.

[0023] In some embodiments, processing device 140 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processing device 140 may be local or remote. For example, processing device 140 may access information and / or data stored in medical scanning device 110, terminal 130, and / or storage device 150 via network 120. As another example, processing device 140 may be directly connected to medical scanning device 110, terminal 130, and / or storage device 150 to access stored information and / or data. In some embodiments, processing device 140 may be implemented on a cloud platform.

[0024] Storage device 150 can store data, instructions, and / or any other information. In some embodiments, storage device 150 can store data obtained from medical scanning device 110, terminal 130, and / or processing device 140. For example, storage device 150 can store medical image data (such as preoperative enhanced images, intraoperative scan images, first segmentation images, second segmentation images, etc.) and / or positioning information data obtained from medical scanning device 110. As another example, storage device 150 can store medical images and / or scanning protocols input from terminal 130. As yet another example, storage device 150 can store data generated by processing device 140 (e.g., medical image data, organ mask data, positioning information data, results data after precise segmentation, spatial location of blood vessels and lesions during surgery, registration maps, etc.). As yet another example, storage device 150 can store data generated by processing device 140 (e.g., segmentation and registration results, interventional risk values, preset weights, weighted risk values, cumulative risk values, image anomaly types, image anomaly degrees, interventional paths, and / or prompt information, etc.).

[0025] In some embodiments, storage device 150 may store data and / or instructions that processing device 140 may execute or be used to execute the exemplary methods described herein. In some embodiments, storage device 150 includes mass storage devices, removable storage devices, volatile read-write memory, read-only memory (ROM), etc., or any combination thereof. Exemplary mass storage devices may include disks, optical disks, solid-state drives, etc. Exemplary removable storage devices may include flash drives, floppy disks, optical disks, memory cards, compact disks, magnetic tapes, etc. Exemplary volatile read-write memory may include random access memory (RAM). Exemplary RAM may include dynamic random access memory (DRAM), double data rate synchronous dynamic access memory (DDR SDRAM), static random access memory (SRAM), thyristor random access memory (T-RAM), and zero-capacitance random access memory (Z-RAM), etc. Exemplary ROM may include masked read-only memory (MROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), optical disc read-only memory (CD-ROM), and digital multifunction disk reallocation memory, etc. In some embodiments, the storage device 150 may be implemented on a cloud platform.

[0026] In some embodiments, storage device 150 may be connected to network 120 to communicate with one or more other components of medical image processing system 100 (e.g., processing device 140, terminal 130). One or more components of medical image processing system 100 may access data or instructions stored in storage device 150 via network 120. In some embodiments, storage device 150 may be directly connected to or communicate with one or more other components of medical image processing system 100 (e.g., processing device 140, terminal 130). In some embodiments, storage device 150 may be part of processing device 140.

[0027] The description of the medical image processing system 100 is intended to be illustrative and not to limit the scope of this specification. Many alternatives, modifications, and variations will be apparent to those skilled in the art. It will be understood that those skilled in the art, upon understanding the principles of the system, may arbitrarily combine the various modules or construct subsystems connected to other modules without departing from these principles. In some embodiments, Figure 21The first acquisition module 910, the second acquisition module 920, the first segmentation module 930, the third acquisition module 940, the second segmentation module 950, the deformation registration module 960, and the risk assessment module 970 can be different modules within a single system, or a single module can implement the functions of two or more of the aforementioned modules. For example, the modules can share a single storage module, or each module can have its own separate storage module. The features, structures, methods, and other features of the exemplary embodiments described in this specification can be combined in various ways to obtain other and / or alternative exemplary embodiments. For example, the processing device 140 and the medical scanning device 110 can be integrated into a single device. Such variations are all within the scope of protection of this specification.

[0028] Figure 2 This is an exemplary flowchart of a method for processing medical images for interventional surgery, provided according to some embodiments of this specification.

[0029] Step 210: Obtain the pattern of the planned interventional path. In some embodiments, step 210 may be performed by the first acquisition module 910 or the medical scanning device 110.

[0030] Interventional surgery, also known as interventional therapy, is a minimally invasive surgical procedure performed using modern high-tech methods. Specifically, under the guidance of medical scanning or imaging equipment, specialized catheters, guidewires, and other precision instruments are introduced into the human body to diagnose and treat internal diseases. In some embodiments, interventional surgery can be an actual (patient-to-patient) diagnostic procedure, or an interventional procedure in an animal experiment or simulation phase; the embodiments in this specification do not impose any particular limitations on it.

[0031] In some embodiments, the interventional path refers to the path taken by instruments used in interventional surgery when introduced into the human body. The interventional path mode may include a precise planning mode and a rapid planning mode. In some embodiments, the precise planning mode or the rapid planning mode may be a path planning mode for segmenting scanned images during surgery. In some embodiments, the precise planning mode may include a precise segmentation mode. In some embodiments, the rapid planning mode may include a rapid segmentation mode.

[0032] In some embodiments, a pattern for planning an interventional path can be obtained. In some embodiments, the pattern for planning an interventional path can be obtained from a medical scanning device 110. In some embodiments, the pattern for planning an interventional path can be obtained from a terminal 130, a processing device 140, and a storage device 150.

[0033] Step 220: Acquire preoperative enhanced images. In some embodiments, step 220 may be performed by the second acquisition module 920 or the medical scanning device 110.

[0034] Preoperative contrast-enhanced images, also known as preoperative contrast-enhanced images, are images obtained by scanning a subject (such as a patient) using medical scanning equipment after a contrast agent has been injected before surgery. In some embodiments, preoperative contrast-enhanced images may include CT images, PET-CT images, US images, or MR images.

[0035] In some embodiments, preoperative enhanced images of the scanned object can be acquired. In some embodiments, preoperative enhanced images of the scanned object, such as PET-CT images, can be acquired from a medical scanning device 110. In some embodiments, preoperative enhanced images of the scanned object, such as US images, can be acquired from a terminal 130, a processing device 140, and a storage device 150.

[0036] In some embodiments, the scanned object may include a biological scanned object or a non-biological scanned object. In some embodiments, a biological scanned object may include a patient, a specific site of the patient, an organ, and / or tissue, such as the abdomen, heart, or tumor tissue. In some embodiments, a non-biological scanned object may include an artificial object, such as a phantom.

[0037] It should be noted that in other embodiments, preoperative enhanced images can also be obtained by any other feasible means, such as obtaining preoperative enhanced images from a cloud server and / or a medical system (such as a hospital's medical system center) via network 120. The embodiments in this specification do not particularly limit this.

[0038] Step 230: Segment the first target structure set in the preoperative enhanced image to obtain a first segmented image of the first target structure set. In some embodiments, step 230 may be performed by the first segmentation module 930.

[0039] In some embodiments, the first target structure set in the preoperative enhanced image may include blood vessels within a target organ (e.g., a target organ). In some embodiments, the first target structure set in the preoperative enhanced image may include the target organ and lesions in addition to blood vessels within the target organ. In some embodiments, the target organ may include the brain, lungs, liver, spleen, kidneys, or any other possible organ tissue, such as the thyroid gland. The first segmented image is a segmented image of the first target structure set obtained by segmenting the preoperative enhanced image (e.g., the target organ, blood vessels within the target organ, and lesions in the preoperative enhanced image).

[0040] Step 240: Acquire intraoperative scan images. In some embodiments, step 240 may be performed by a third acquisition module 940.

[0041] Intraoperative scan images refer to images obtained by scanning the subject using a medical scanning device during surgery. In some embodiments, intraoperative scan images may include CT images, PET-CT images, US images, or MR images. In some embodiments, intraoperative scan images may be real-time scan images. In some embodiments, intraoperative scan images may also be referred to as preoperative plain scan images or intraoperative plain scan images, which are scan images taken during surgical preparation and before the actual surgical procedure (i.e., before needle insertion).

[0042] In some embodiments, intraoperative scan images of the subject can be acquired. In some embodiments, intraoperative scan images of the subject, such as PET-CT images, can be acquired from a medical scanning device 110. In some embodiments, intraoperative scan images of the subject, such as US images, can be acquired from a terminal 130, a processing device 140, and a storage device 150.

[0043] Step 250: Segment the second target structure set in the surgical scan image to obtain a second segmented image of the second target structure set. In some embodiments, step 250 may be performed by the second segmentation module 950.

[0044] In some embodiments, the regions or organs included in the second target structure set of intraoperative scan images can be determined based on the mode of interventional path planning (e.g., rapid planning mode and precise planning mode). That is, the regions or organs included in the second target structure set differ depending on the mode of interventional path planning. For example, in rapid planning mode, the second target structure set may include non-invasive regions. As another example, in precise planning mode, the second target structure set may include all vital organs in the intraoperative scan images. Vital organs refer to organs that the interventional path planning needs to avoid during interventional surgery, such as the liver, kidneys, and external blood vessels of the target organ. In some embodiments, the second target structure set may also include target organs and lesions in addition to non-invasive regions / all vital organs in the intraoperative scan images. The second segmented image is a segmented image of the second target structure set (e.g., non-invasive regions / vital organs, target organs, lesions) obtained by segmenting the intraoperative scan images.

[0045] In some embodiments, the first target structure set and the second target structure set have an intersection. For example, when the first target structure set includes blood vessels within the target organ and the target organ itself, and the second target structure set includes a non-invasive region (or all vital organs), the target organ, and the lesion, the intersection of the first and second target structure sets is the target organ. As another example, when the first target structure set includes blood vessels within the target organ, the target organ, and the lesion, and the second target structure set includes a non-invasive region (or all vital organs), the target organ, and the lesion, the intersection of the first and second target structure sets is the target organ and the lesion.

[0046] In some embodiments, step 250, segmenting the target organ in the intraoperative scan image, can be implemented as follows: segmenting a second set of target structures in the intraoperative scan image according to a planning mode. In some embodiments, a fourth set of target structures in the intraoperative scan image can be segmented according to a fast segmentation mode and / or a precise segmentation mode.

[0047] In some embodiments, the fourth target structure set may be part of the second target structure set, such as non-invasive regions or all important organs outside the target organ. The regions / organs included in the fourth target structure set differ depending on the planning mode. In some embodiments, in rapid segmentation mode, the fourth target structure set may include non-invasive regions. In some embodiments, in precise segmentation mode, the fourth target structure set may include predefined important organs.

[0048] In some embodiments, in the rapid segmentation mode, regional localization calculations can be performed on the intraoperative scan images, and non-invasive regions can be segmented and extracted.

[0049] Non-invasive areas refer to areas that the planned interventional path must avoid during interventional procedures. In some embodiments, non-invasive areas may include non-puncture areas, non-insertion or implantation areas, and non-injection areas.

[0050] In some embodiments, post-processing can be performed on regions outside the non-invasive region and the target organ (e.g., the target organ) to ensure that there are no void regions between the non-invasive region and the target organ. A void region refers to a background region surrounded by a boundary connected to foreground pixels. In some embodiments, the non-invasive region can be obtained by subtracting the target organ and the invasive region from the abdominal (or thoracic) region. After subtracting the target organ and the invasive region from the abdominal (or thoracic) region to obtain the non-invasive region, a void region may exist between the target organ and the non-invasive region. This void region belongs neither to the target organ nor to the non-invasive region. In this case, post-processing operations can be performed on the void region to fill it in; that is, the void region after post-processing can be considered a non-invasive region. In some embodiments, post-processing can include erosion and dilation operations. In some embodiments, erosion and dilation operations can be implemented based on convolution processing of intraoperative scan images and filters. In some embodiments, the erosion operation may involve convolving a filter with the intraoperative scan image and then finding a local minimum based on a predetermined erosion range, thereby reducing the outline of the intraoperative scan image to a desired range and shrinking the target highlight area in the initial image displayed in the intraoperative scan image by a certain range. In some embodiments, the dilation operation may involve convolving a filter with the intraoperative scan image and then finding a local maximum based on a predetermined erosion range, thereby expanding the outline of the intraoperative scan image to a desired range and shrinking the target highlight area in the initial image displayed in the intraoperative scan image by a certain range.

[0051] In some embodiments, in the rapid segmentation mode, region localization calculations can be performed on the intraoperative scan images before segmentation and extraction of non-invasive regions. In some embodiments, the vascular mask inside the target organ can be determined based on the segmentation mask and vascular mask of the target organ in the intraoperative scan images. It should be noted that in the rapid segmentation mode, only the blood vessels inside the target organ need to be segmented; in the precise segmentation mode, both the blood vessels inside the target organ and other external blood vessels can be segmented.

[0052] A mask, such as an organ mask, can be a pixel-level classification label. Taking abdominal medical images as an example, a mask represents the classification of individual pixels in the medical image. For example, it can be divided into background, liver, spleen, kidney, etc. The summary area of ​​a specific category is represented by the corresponding label value. For example, all pixels classified as liver are summarized, and the summary area is represented by the label value corresponding to liver. The label value here is set according to the specific coarse segmentation task. A segmentation mask refers to the corresponding mask obtained after the segmentation operation. In some embodiments, the mask can include an organ mask (such as an organ mask of the target organ) and a blood vessel mask.

[0053] In some embodiments, using the rapid segmentation mode as an example, taking only the thoracic or abdominal cavity region, the localization of the thoracic or abdominal cavity region within the scanning range of the surgical scan image is first calculated. Specifically, for the abdominal cavity, the area from the top of the liver to the bottom of the rectum is selected as the localization area; for the thoracic cavity, the area from the top of the esophagus to the bottom of the lung (or the top of the liver) is selected as the localization area. After determining the localization information of the thoracic or abdominal cavity region, the abdominal or thoracic cavity is segmented, and further segmentation is performed within this segmented area to extract the interventional area (as opposed to the non-interventional area, such as the perforable area, fat, etc.). Finally, the segmentation mask of the target organ and the perforable area mask are removed using the abdominal cavity segmentation mask to extract the non-interventional area. In some embodiments, the interventional area may include a fatty portion, such as the gap between two organs containing fat. Taking the liver as an example, a portion of the area between the subcutaneous tissue and the liver may be covered by fat. Because the rapid segmentation mode has a fast processing speed, the planning speed is faster and the time is shorter, improving the image processing efficiency.

[0054] In some embodiments, in precise segmentation mode, all organs in the intraoperative scan image can be segmented. In some embodiments, all organs in the intraoperative scan image may include basic organs and important organs. In some embodiments, basic organs in the intraoperative scan image may include target organs (e.g., target organs). In some embodiments, in precise segmentation mode, preset important organs in the intraoperative scan image can be segmented. Preset important organs can be determined based on the importance of each organ in the intraoperative scan image. For example, all important organs in the intraoperative scan image can be used as preset important organs. In some embodiments, the ratio of the total volume of preset important organs in rapid segmentation mode to the total volume of preset important organs in precise segmentation mode can be greater than a preset efficiency factor m. The preset efficiency factor m can characterize the difference in segmentation efficiency (or segmentation detail) based on different segmentation modes. In some embodiments, the preset efficiency factor m can be equal to or greater than 1. In some embodiments, the setting of the efficiency factor m is related to the type of interventional surgery. Interventional surgery types may include, but are not limited to, urological surgery, thoracic and abdominal surgery, cardiovascular surgery, obstetric and gynecological interventional surgery, and musculoskeletal surgery. For illustrative purposes only, the preset efficiency factor m in urological surgery can be set to a larger value; the preset efficiency factor m in thoracic and abdominal surgery can be set to a smaller value.

[0055] In some embodiments, in precise segmentation mode, segmentation masks for all organs in the surgical scan images are obtained through segmentation. In some embodiments, in precise segmentation mode, segmentation masks and vascular masks for all organs in the surgical scan images are obtained through segmentation. In some embodiments, in precise segmentation mode, vascular masks within all organs are determined based on the segmentation masks and vascular masks for all organs in the surgical scan images. Therefore, in precise segmentation mode, the segmented image content is more detailed, resulting in more selectivity in path planning and enhanced robustness of image processing.

[0056] Figure 3 This is an exemplary flowchart illustrating the segmentation process involved in an interventional surgical medical image processing method according to some embodiments of this specification. Figure 3 As shown, process 300 may include the following steps: Step 310: Perform coarse segmentation on at least one element of the target structure set in the medical image; Step 320: Obtain a mask for at least one element; Step 330: Determine the mask's positioning information; Step 340: Based on the positioning information of the mask, accurately segment at least one element.

[0057] In some embodiments, medical images may include preoperative enhanced images and intraoperative scans. The target structure set may include any one or more of the first target structure set, the second target structure set, and the fourth target structure set.

[0058] In some embodiments, step 310 may utilize a threshold segmentation method, a region growing method, or a level set method to perform coarse segmentation on at least one element of the target structure set in the medical image. Elements may include target organs (e.g., target organs), blood vessels within the target organ, lesions, non-invasive areas, all important organs, etc., in the medical image. In some embodiments, coarse segmentation based on the threshold segmentation method may be implemented as follows: by setting multiple different pixel threshold ranges, each pixel in the medical image is classified according to the pixel value of the input medical image, and pixels with pixel values ​​within the same pixel threshold range are segmented into the same region. In some embodiments, coarse segmentation based on the region growing method may be implemented as follows: based on known pixels or predetermined regions composed of pixels in the medical image, a preset similarity discrimination condition is established according to requirements, and based on this preset similarity discrimination condition, pixels are compared with their surrounding pixels, or predetermined regions are compared with their surrounding regions, merging pixels or regions with high similarity until the above process cannot be repeated, at which point merging stops, completing the coarse segmentation process. In some embodiments, the preset similarity discrimination condition may be determined based on preset image features, such as grayscale, texture, and other image features. In some embodiments, coarse segmentation based on the level set method can be implemented as follows: the target contour of the medical image is set as the zero level set of a high-dimensional function, the function is differentiated, the zero level set is extracted from the output to obtain the contour of the target, and then the pixel region within the contour range is segmented.

[0059] In some embodiments, a method based on deep learning convolutional networks can be used to perform coarse segmentation on at least one element of a target structure set in a medical image. In some embodiments, the method based on deep learning convolutional networks may include a segmentation method based on fully convolutional networks. In some embodiments, the convolutional network may employ a U-shaped network framework, such as UNet. In some embodiments, the network framework of the convolutional network may consist of an encoder and a decoder, as well as a skip connection structure. The encoder and decoder are composed of convolutional layers or convolutional layers combined with an attention mechanism. The convolutional layers are used to extract features, the attention module is used to apply more attention to key regions, and the skip connection structure is used to combine features extracted by the encoder in different dimensions into the decoder. Finally, the segmentation result is output by the decoder. In some embodiments, coarse segmentation based on deep learning convolutional networks can be implemented as follows: the encoder of the convolutional neural network extracts features from the medical image through convolution, and then the decoder of the convolutional neural network restores the features into a pixel-level segmentation probability map. The segmentation probability map represents the probability that each pixel in the image belongs to a specific category. Finally, the segmentation probability map is output as a segmentation mask, thereby completing the coarse segmentation.

[0060] Figure 4 This is an exemplary flowchart illustrating the process of determining the positioning information of an element mask according to some embodiments of this specification. Figure 5 This is an exemplary flowchart of a soft connected domain analysis process using element masks as shown in some embodiments of this specification. Figure 6 This is a comparison diagram of exemplary effects of coarse segmentation of element masks for soft connected domain analysis, as shown in some embodiments of this specification.

[0061] In some embodiments, in step 330, determining the positioning information of the element mask can be implemented as follows: performing soft connected component analysis on the element mask. A connected component, or connected region, generally refers to an image region composed of foreground pixels with the same pixel value and adjacent positions.

[0062] In some embodiments, step 330, which performs soft connected component analysis on the element mask, may include the following sub-steps: Sub-step 331: Determine the number of connected components; Sub-step 332: When the number of connected components is ≥2, determine the area of ​​the connected components that meet the preset conditions; Sub-step 333: When the ratio of the area of ​​the largest connected component to the total area of ​​the connected components is greater than the first threshold M, the largest connected component is determined to meet the preset condition. Sub-step 334: Determine that the remaining connected components include at least the largest connected component; Sub-step 335: Determine the positioning information of the element mask based on the preserved connected components.

[0063] Preset conditions refer to the conditions that a connected component must meet to be retained as a retained connected component. For example, a preset condition could be a limitation on the area of ​​the connected component. In some embodiments, a medical image may include multiple connected components with different areas. These multiple connected components with different areas can be sorted according to their area size, for example, from largest to smallest. The sorted connected components can be denoted as the first connected component, the second connected component, and the k-th connected component. The first connected component can be the connected component with the largest area among the multiple connected components, also called the largest connected component. In this case, the preset conditions for determining whether a connected component with different area rankings is a retained connected component can be different; see [link to relevant documentation] for details. Figure 5The relevant description is as follows. In some embodiments, connected components that meet preset conditions may include connected components whose areas are sorted from largest to smallest within a preset order n. For example, when the preset order n is 3, each connected component can be judged sequentially according to its area order and the corresponding preset conditions to determine whether it is a retained connected component. That is, the first connected component is judged to determine whether it is a retained connected component first, and then the second connected component is judged to determine whether it is a retained connected component. In some embodiments, the preset order n can be set based on the category of the target structure, such as a chest target structure or an abdominal target structure. In some embodiments, the value range of the first threshold M can be 0.8 to 0.95, which can ensure that the soft connected component analysis obtains the expected accuracy within the value range. In some embodiments, the value range of the first threshold M can be 0.9 to 0.95, which further improves the accuracy of the soft connected component analysis. In some embodiments, the first threshold M can be set based on the category of the target structure, such as a chest target structure or an abdominal target structure. In some embodiments, the preset order n / first threshold M can also be reasonably set according to machine learning and / or big data, which is not further limited here.

[0064] In some embodiments, step 330, which performs soft connected component analysis on the element mask, can be done as follows: Based on the obtained element mask, the number of connected components within the element mask and their corresponding areas are analyzed and calculated. The process is as follows: When the number of connected components is 0, it indicates that the corresponding mask is empty, meaning that mask acquisition or coarse segmentation failed or the segmentation object does not exist, and no processing is performed. For example, when segmenting the spleen in the abdominal cavity, there may be cases where the spleen has been removed, in which case the spleen mask is empty.

[0065] When the number of connected components is 1, it means that there is only one connected component, no false positives, no splits or breaks, and the connected component is retained. It can be understood that when the number of connected components is 0 or 1, there is no need to determine whether the connected component is a retained connected component based on preset conditions.

[0066] When there are two connected components, connected components A and B are obtained based on their areas (S), where the area of ​​connected component A is greater than the area of ​​connected component B, i.e., S(A) > S(B). Based on the above, connected component A can also be called the first connected component or the largest connected component; connected component B can be called the second connected component. When there are two connected components, the preset condition for a connected component to be retained can be the relationship between the ratio of the area of ​​the largest connected component to the total area of ​​the connected components and a threshold. Calculating the connected components, when the proportion of the area of ​​A to the total area of ​​A and B is greater than the first threshold M, i.e., S(A) / S(A+B) > the first threshold M, connected component B can be determined as a false positive region, and only connected component A is retained (i.e., connected component A is determined as a retained connected component); when the proportion of the area of ​​A to the total area of ​​A and B is less than or equal to the first threshold M, both A and B can be determined as part of the element mask, and both connected components A and B are retained (i.e., connected components A and B are determined as retained connected components).

[0067] When the number of connected components is greater than or equal to 3, connected components A, B, C...P are obtained according to their area (S), where the area of ​​connected component A is greater than the area of ​​connected component B, the area of ​​connected component B is greater than the area of ​​connected component C, and so on, i.e., S(A)>S(B)>S(C)>...>S(P). Then, the total area S(T) of connected components A, B, C...P is calculated. At this time, the connected components can be judged according to the area order and according to the corresponding preset conditions to determine whether each connected component (or the connected component whose area order is within the preset order n) is a retained connected component. In some embodiments, when the number of connected components is greater than or equal to 3, the preset condition that the largest connected component (i.e., connected component A) needs to satisfy as a retained connected component can be the relationship between the ratio of the area of ​​the largest connected component to the total area of ​​the connected components and a threshold (e.g., a first threshold M). In some embodiments, when the number of connected components is greater than or equal to 3, the preset condition that the largest connected component (i.e., connected component A) needs to satisfy as a retained connected component can also be the relationship between the ratio of the area of ​​the second connected component to the area of ​​the largest connected component and a threshold (e.g., the second threshold N). Specifically, when the proportion of the area of ​​connected component A to the total area S(T) is greater than the first threshold M, i.e., S(A) / S(T)>the first threshold M, or when the proportion of the area of ​​connected component B to the area of ​​connected component A is less than the second threshold N, i.e., S(B) / S(A)<the second threshold N, connected component A is determined as the element mask part and retained (i.e., connected component A is a retained connected component), and the remaining connected components are determined as false positive regions; otherwise, the calculation continues, i.e., the determination of whether the second connected component (i.e., connected component B) is a retained connected component continues. In some embodiments, the preset condition that connected component B needs to satisfy as a retained connected component can be the relationship between the ratio of the sum of the areas of the first connected component and the second connected component to the total area of ​​the connected components and the first threshold M. In some embodiments, the preset condition that connected component B must satisfy to be considered a retained connected component can also be the relationship between the proportion of the area of ​​the third connected component to the sum of the areas of the first and second connected components and a threshold (e.g., a second threshold N). Specifically, when the proportion of the areas of connected components A and B to the total area S(T) is greater than the first threshold M, i.e., S(A+B) / S(T)>the first threshold M, or when the proportion of the area of ​​connected component C to the areas of connected components A and B is less than the second threshold N, i.e., S(C) / S(A+B)<the second threshold N, connected components A and B are determined to be the element mask portion and retained (i.e., connected components A and B are retained connected components), and the remaining parts are all determined to be false positive regions; otherwise, the calculation continues, i.e., the determination of whether the third connected component (i.e., connected component C) is a retained connected component continues.The method for determining connected component C is similar to that for connected component B. The preset condition for connected component C to be retained as a connected component can be the relationship between the ratio of the sum of the areas of the first, second, and third connected components to the total area of ​​the connected components and the first threshold M, or the relationship between the proportion of the area of ​​the fourth connected component to the sum of the areas of the first, second, and third connected components and a threshold (e.g., the second threshold N). Specifically, when the proportion of the area of ​​connected components A, B, and C to the total area S(T) is greater than the first threshold M (i.e., S(A+B+C) / S(T)>the first threshold M), or when the proportion of the area of ​​connected component D to the area of ​​connected components A, B, and C is less than the second threshold N (i.e., S(D) / S(A+B+C)<the second threshold N), connected components A, B, and C are all determined as element mask portions and retained (i.e., connected components A, B, and C are all retained connected components). Following the above judgment method, we can sequentially determine whether connected components A, B, C, D…P, or connected components whose area order is within a preset order n, are retained connected components. It should be noted that… Figure 4 The text only shows the determination of whether three connected components are preserved connected components. This can also be understood as... Figure 4 The preset value of the sequence number n is set to 4. Therefore, it is only necessary to determine whether the connected components with sequence numbers 1, 2, and 3, namely connected component A, connected component B, and connected component C, are reserved connected components.

[0068] Finally, output the preserved connected components.

[0069] In some embodiments, the second threshold N can range from 0.05 to 0.2, which ensures that the soft connected component analysis achieves the expected accuracy. In some embodiments, the second threshold N can range from 0.05, which achieves better accuracy in soft connected component analysis.

[0070] like Figure 6 As shown, the top and bottom left images are cross-sectional and stereoscopic medical images with coarse segmentation results without soft connected component analysis, while the right images are cross-sectional and stereoscopic medical images with coarse segmentation results using soft connected component analysis. The comparison reveals that the coarse segmentation of the element mask based on soft connected component analysis removes the false positive regions outlined by the bounding boxes in the left image. Compared to previous connected component analysis methods, this method demonstrates higher accuracy and reliability in eliminating false positive regions and directly contributes to the subsequent extraction of bounding boxes for element mask localization information, thus improving segmentation efficiency.

[0071] In some embodiments, the positioning information of the element mask can be the position information of the outer rectangle of the element mask, such as the coordinate information of the border line of the outer rectangle. In some embodiments, the outer rectangle of the element mask covers the positioning area of ​​the element. In some embodiments, the outer rectangle can be displayed in the medical image in the form of an outer bounding box. In some embodiments, the outer rectangle can be constructed relative to the element mask based on the bottom edges of the connected regions in each direction of the element, such as the bottom edges of the connected regions in the top, bottom, left, and right directions.

[0072] In some embodiments, the bounding rectangle of the element mask can be a single rectangle or a combination of multiple rectangles. For example, it can be a large rectangle or a larger rectangle formed by combining multiple smaller rectangles.

[0073] In some embodiments, the bounding rectangle of the element mask can be a bounding rectangle that contains only one bounding rectangle. For example, when there is only one connected region in the element (e.g., a blood vessel or an organ in the abdominal cavity), a larger bounding rectangle is constructed based on the bottom edges of the connected region in each direction. In some embodiments, the aforementioned large bounding rectangle can be applied to organs containing only one connected region.

[0074] In some embodiments, the outer rectangle of an element mask can be a single outer rectangle composed of multiple rectangles. For example, when an element has multiple connected regions, the multiple rectangles corresponding to the multiple connected regions are combined into a single rectangle based on their bottom edges. It is understandable that, for instance, the bottom edges of the three rectangles corresponding to three connected regions can be combined into a single outer rectangle, and the calculation is performed using this single outer rectangle, thus reducing computational complexity while maintaining the expected accuracy.

[0075] In some embodiments, when a medical image includes multiple connected components, the positional information of the multiple connected components can be determined first, and then the positioning information of the element mask can be obtained based on the positional information of the multiple connected components. For example, the connected components that meet preset conditions can be determined first, that is, the positional information of the connected components can be retained, and then the positioning information of the element mask can be obtained based on the retained positional information of the connected components.

[0076] In some embodiments, step 330, determining the positioning information of the element mask, may further include the following operation: positioning the element mask based on preset element positioning coordinates.

[0077] In some embodiments, this operation can be performed if the positioning of the outer rectangle of the element mask fails. It is understood that if the coordinates of the outer rectangle of the element mask do not exist, the positioning of the corresponding element is considered to have failed.

[0078] In some embodiments, the preset elements can be selected from elements with relatively stable positioning (e.g., organs with relatively stable positioning), where the probability of positioning failure is low, thereby achieving precise positioning of the element mask. In some embodiments, since the probability of positioning failure for the liver, stomach, spleen, and kidneys in the abdominal cavity is low, and the probability of positioning failure for the lungs in the thoracic cavity is low, the positioning of these organs is relatively stable. Therefore, the liver, stomach, spleen, and kidneys can be used as preset organs in the abdominal cavity; that is, the preset elements can include the liver, stomach, spleen, kidneys, lungs, or any other possible organ tissue. In some embodiments, the organ mask in the abdominal cavity can be repositioned based on the positioning coordinates of the liver, stomach, spleen, and kidneys. In some embodiments, the organ mask in the thoracic cavity can be positioned based on the positioning coordinates of the lungs.

[0079] In some embodiments, the element mask can be repositioned using preset element positioning coordinates as reference coordinates. In some embodiments, when the element that failed to be positioned is located in the abdominal cavity, the positioning coordinates of the liver, stomach, spleen, and kidney are used as the repositioning coordinates to reposition the element that failed to be positioned in the abdominal cavity. In some embodiments, when the element that failed to be positioned is located in the thoracic cavity, the positioning coordinates of the lung are used as the repositioning coordinates to reposition the element that failed to be positioned in the thoracic cavity. As an example only, when the element that failed to be positioned is located in the abdominal cavity, the positioning coordinates of the top of the liver, the bottom of the kidney, the left side of the spleen, and the right side of the liver can be used as the coordinates of the transverse (upper and lower) and coronal (left and right) directions for repositioning, and the front and rearmost points of these four organ coordinates can be taken as the coordinates of the sagittal (anterior and posterior) directions for new positioning to reposition the element that failed to be positioned in the abdominal cavity. As an example only, when an element that failed to be located is located within the chest cavity, the bounding rectangle formed by the lung positioning coordinates is expanded by a certain number of pixels, and the element that failed to be located in the chest cavity is repositioned accordingly.

[0080] Precise positioning of element masks based on preset element coordinates improves segmentation accuracy and reduces segmentation time, thereby increasing segmentation efficiency. It also reduces the amount of segmentation computation and saves memory resources.

[0081] Figure 7 This is an exemplary flowchart illustrating the process of accurately segmenting elements according to some embodiments of this specification.

[0082] In some embodiments, step 340, based on the positioning information of the mask, accurately segmenting at least one element may include the following sub-steps: Sub-step 341 involves performing preliminary precise segmentation on at least one element. This preliminary precise segmentation can be based on the positioning information of the coarsely segmented element mask. In some embodiments, the element can be preliminarily precisely segmented based on the input data and the bounding rectangle of the coarsely segmented positioning. This preliminary precise segmentation generates a precisely segmented element mask.

[0083] Sub-step 342 determines whether the positioning information of the element mask is accurate. Step 342 determines whether the positioning information of the element mask obtained from the coarse segmentation is accurate, and further determines whether the coarse segmentation is accurate.

[0084] In some embodiments, the positioning information of the initially precisely segmented element mask can be calculated, and the positioning information of the coarse segmentation can be compared with the positioning information of the precisely segmented mask. In some embodiments, the bounding rectangle of the coarse segmented element mask can be compared with the bounding rectangle of the precisely segmented element mask to determine the magnitude of their difference. In some embodiments, the bounding rectangle of the coarse segmented element mask can be compared with the bounding rectangle of the precisely segmented element mask in six directions in three-dimensional space (i.e., the entire bounding rectangle is a cube in three-dimensional space) to determine the magnitude of their difference. As an example only, the overlap degree of each side of the bounding rectangle of the coarse segmented element mask with each side of the bounding rectangle of the precisely segmented element mask can be calculated, or the difference between the coordinates of the six vertices of the bounding rectangle of the coarse segmented element mask and the bounding rectangle of the precisely segmented element mask can be calculated.

[0085] In some embodiments, the accuracy of the positioning information of the coarsely segmented element mask can be determined based on the positioning information of the initially precisely segmented element mask. In some embodiments, the accuracy of the determination result can be determined based on the difference between the positioning information of the coarse segmentation and the positioning information of the precise segmentation. In some embodiments, the positioning information can be the circumscribed rectangle (e.g., the bounding box) of the element mask. The accuracy of the circumscribed rectangle of the coarsely segmented element mask is determined based on the difference between the circumscribed rectangle of the coarsely segmented element mask and the circumscribed rectangle of the precise segmentation element mask. In this case, the difference between the positioning information of the coarse segmentation and the positioning information of the precise segmentation can refer to the distance between the nearest border lines in the circumscribed rectangle of the coarse segmentation and the circumscribed rectangle of the precise segmentation. In some embodiments, when the difference between the positioning information of the coarse segmentation and the positioning information of the precise segmentation is large (i.e., the distance between the nearest border lines in the circumscribed rectangle of the coarse segmentation and the circumscribed rectangle of the precise segmentation is large), the positioning information of the coarse segmentation is determined to be accurate; when the difference is small (i.e., the distance between the nearest border lines in the circumscribed rectangle of the coarse segmentation and the circumscribed rectangle of the precise segmentation is small), the positioning information of the coarse segmentation is determined to be inaccurate. It is important to note that the coarse segmentation bounding rectangle is obtained by expanding the border lines of the original coarse segmentation elements by pixels (e.g., by 15-20 voxels). In some embodiments, the accuracy of the coarse segmentation's positioning information can be determined based on the relationship between the distance between the nearest border lines in the coarse and precise segmentation bounding rectangles and a preset threshold. For example, a distance less than the preset threshold is considered inaccurate, while a distance greater than the preset threshold is considered accurate. In some embodiments, to ensure accuracy, the preset threshold can be less than or equal to 5 voxels.

[0086] Figures 8 to 9 This is an exemplary schematic diagram showing the determination of the positioning information of the element mask according to some embodiments of this specification. Figure 10A This is an exemplary example diagram illustrating the determination of the sliding direction based on the positioning information of an element mask, as shown in some embodiments of this specification.

[0087] in, Figure 8 , Figure 9 The image shows the element mask A obtained from coarse segmentation, the bounding rectangle B of element mask A (i.e., the positioning information of element mask A), and the bounding rectangle C after the initial precise segmentation based on the bounding rectangle of coarse segmentation. Figure 10AThe diagram also shows the sliding window B1 obtained after sliding the coarsely segmented circumscribed rectangle B. (a) is a schematic diagram before the sliding operation, and (b) is a schematic diagram after the sliding operation. Furthermore, for convenience, an example is given using a planar rectangle within one plane of the 3D circumscribed rectangle. It can be understood that the 3D circumscribed rectangle also contains five other planar rectangles, meaning there are six border lines in six directions during the specific calculation of the 3D circumscribed rectangle. Here, only the four border lines of a single plane are used for illustration.

[0088] For example only, such as Figure 8 As shown, the right border line of the precisely segmented outer rectangle C is only slightly different from the corresponding border line of the coarsely segmented outer rectangle B. This indicates that the right side of the coarsely segmented outer rectangle B is inaccurate in the corresponding direction and needs adjustment. However, the top, bottom, and left border lines of C differ significantly from the top, bottom, and left border lines of B. This indicates that the top, bottom, and left border lines of the coarsely segmented outer rectangle B are accurate in the corresponding directions. This is just an example. Figure 9 As shown, the border lines of the four sides of the precisely segmented outer rectangle C differ significantly from the corresponding border lines of the coarsely segmented outer rectangle B. This indicates that the border lines of the four sides of the coarsely segmented outer rectangle B are accurate. It should be noted that the element mask A has six directions. Figure 8 , Figure 9 The illustration only uses 4 border lines, but in reality, 12 border lines in 6 directions in element mask A will be judged.

[0089] Sub-step 343a: If the judgment result is inaccurate, obtain accurate positioning information based on an adaptive sliding window. In some embodiments, when the coarse segmentation result is inaccurate, the elements obtained by its precise segmentation are likely to be inaccurate as well. Therefore, appropriate adaptive sliding window calculations can be performed to obtain accurate positioning information for continued precise segmentation.

[0090] In some embodiments, obtaining accurate positioning information based on adaptive sliding windows can be implemented as follows: determining at least one direction where the positioning information is inaccurate; performing adaptive sliding window calculation in said direction according to an overlap rate parameter. In some embodiments, at least one direction where the circumscribed rectangle is inaccurate can be determined; after determining that the coarsely segmented circumscribed rectangle is inaccurate, the coarsely segmented circumscribed rectangle is slid in the corresponding direction according to the input preset overlap rate parameter, i.e., a sliding window operation is performed, and this sliding window operation is repeated until all circumscribed rectangles are completely accurate. The overlap rate parameter refers to the ratio of the area of ​​the overlapping portion between the initial circumscribed rectangle and the slidable circumscribed rectangle to the area of ​​the initial circumscribed rectangle. When the overlap rate parameter is high, the sliding step size of the sliding window operation is short. In some embodiments, to ensure a simpler sliding window calculation process (i.e., fewer steps in the sliding window operation), the overlap rate parameter can be set smaller; to ensure a more accurate sliding window calculation result, the overlap rate parameter can be set larger. In some embodiments, the sliding step size for the sliding window operation can be calculated based on the current overlap rate parameter. Figure 8 According to the judgment method, Figure 10A The directions corresponding to the right and bottom border lines of the circumscribed rectangle B with a medium-thickness division are inaccurate. For ease of description, we will denote the direction corresponding to the right border line of the circumscribed rectangle B as the first direction (the first direction is perpendicular to the right border line of B), and the direction corresponding to the bottom border line as the second direction (the second direction is perpendicular to the bottom border line of B). This is only for example. Figure 10A As shown, assuming the length of the circumscribed rectangle B is 'a', when the overlap ratio parameter is 60%, the corresponding step size can be determined to be 'a'. (1-60%), as described above, the right border of the circumscribed rectangle B can slide along the first direction a. (1-60%). Similarly, the bottom border of the circumscribed rectangle B can slide along the second direction with corresponding steps. Repeat the corresponding sliding window operation for the right and bottom borders of the circumscribed rectangle B until the circumscribed rectangle B is completely accurate, as shown below. Figure 10A (b) shows the sliding window B1. Combined with... Figure 8 and Figure 10A When it is determined that the outer bounding rectangle of the coarse segmentation (i.e., the positioning information of the target structure mask) is inaccurate, the coordinate values ​​of the border lines in the six directions of the fine segmentation outer bounding rectangle are compared one by one with the coordinate values ​​of the border lines in the six directions of the coarse segmentation outer bounding rectangle. When the difference is less than the coordinate difference threshold (for example, the coordinate difference threshold is 5pt) (where the coordinate difference threshold can be set according to the actual situation and is not limited here), it can be determined that the border line of the outer bounding rectangle is inaccurate.

[0091] For example, such as Figure 8As shown, the pixel coordinates of the four sides in the finely segmented bounding rectangle C image in four directions are compared one by one with the pixel coordinates of the four border lines in the coarsely segmented bounding rectangle B image in four directions. When the difference in pixel coordinates in one direction is less than the coordinate difference threshold of 8pt, it can be determined that... Figure 8 The direction of the coarsely divided bounding rectangle is inaccurate. For example, if the difference between the top and bottom edges is 20pt, the difference between the bottom and right edges is 30pt, the difference between the right and left edges is 1pt, and the difference between the left and right edges is 50pt. In this case, the direction corresponding to the right edge is inaccurate, while the directions corresponding to the top, bottom, and left edges are accurate.

[0092] For example, combining Figure 10A Here, B1 is the outer rectangle (also called the sliding window) obtained after sliding the coarsely segmented outer rectangle B. The sliding window is a coarsely segmented outer rectangle that meets the expected accuracy standard. The border lines of the coarsely segmented outer rectangle B (e.g., the right border line and the bottom border line) need to be slid along corresponding directions (e.g., the first direction and the second direction) by the corresponding step length to the position of the sliding window B1. Specifically, each non-standard border line is moved sequentially in its corresponding direction; for example, the right border line of B is slid first, then the bottom border line of B is slid to the specified position of the sliding window. The left and top borders of B are standard and do not need to be slid. The step length for each side depends on the overlap rate between B1 and B. The overlap rate can be the ratio of the current overlapping area of ​​the coarsely segmented outer rectangle B and the sliding window B1 to the total area; for example, the current overlap rate is 40%, etc. It should be noted that the sliding order of the border lines of the coarsely segmented outer rectangle B can be from left to right, from top to bottom, or other feasible orders; no further limitation is made here.

[0093] Figures 10B-10E This is an exemplary schematic diagram illustrating precise segmentation after sliding a window, as shown in some embodiments of this specification. Combined with... Figure 10B-10E In some embodiments, based on the original coarsely segmented bounding rectangle (original sliding window), an accurate coarsely segmented bounding rectangle is obtained after adaptive sliding window. This allows for the acquisition of accurate bounding rectangle coordinates. Based on these coordinates and the overlap rate parameter, the new sliding window is precisely segmented. The precise segmentation result is then superimposed with the preliminary precise segmentation result to obtain the final precise segmentation result. Specifically, see [link to documentation]. Figure 10B A sliding window operation can be performed on the original sliding window B to obtain sliding window B1 (the bounding rectangle of the maximum area after the sliding window operation). B is then slid along the first direction by a corresponding step to obtain sliding window B1-1. Finally, the entire area of ​​sliding window B1-1 is precisely segmented to obtain the precise segmentation result of sliding window B1-1. Further details can be found in [link to documentation]. Figure 10CB can be slid along the second direction with a corresponding step size to obtain sliding window B1-2. Then, the entire range of sliding window B1-2 is precisely segmented to obtain the precise segmentation result of sliding window B1-2. Further, see... Figure 10D Sliding B will create sliding windows B1-3 (e.g., B can be adjusted according to...). Figure 10C The sliding operation shown yields sliding window B1-2, which is then slid down to obtain sliding window B1-3. The entire area of ​​sliding window B1-3 is then precisely segmented to obtain the precise segmentation result. The precise segmentation results of sliding windows B1-1, B1-2, and B1-3 are then superimposed with the preliminary precise segmentation result to obtain the final precise segmentation result. It should be noted that the dimensions of sliding windows B1-1, B1-2, and B1-3 are the same as the dimension of B. Sliding window B1 is the final sliding window result obtained by performing continuous sliding operations on the original sliding window B, i.e., sliding windows B1-1, B1-2, and B1-3. In some embodiments, when the precise segmentation results of sliding windows B1-1, B1-2, and B1-3 are superimposed with the preliminary precise segmentation result, there may be overlapping parts, for example... Figure 10E In the image, there may be an overlap between sliding windows B1-1 and B1-2. When overlaying the segmentation results, this overlap may be repeatedly overlaid. To address this, the following method can be used: For a portion of element mask A, if one sliding window's segmentation result is accurate while the other's is inaccurate, the segmentation result from the accurate window is used as the segmentation result for that portion. If both sliding windows' segmentation results are accurate, the segmentation result from the right sliding window is used as the segmentation result for that portion. If neither sliding window's segmentation result is accurate, the segmentation result from the right sliding window is used as the segmentation result for that portion, and precise segmentation continues until the segmentation result is accurate.

[0094] In some embodiments, such as Figure 7As shown, when the judgment result is inaccurate, obtaining accurate positioning information based on the adaptive sliding window is a cyclical process. Specifically, after comparing the precise segmentation border line and the coarse segmentation border line, the coordinate values ​​of the updated precise segmentation bounding rectangle can be obtained through the adaptive sliding window. This precise segmentation bounding rectangle is expanded by a certain number of pixels and set as the coarse segmentation bounding rectangle for the next round of the cycle. Then, the new bounding rectangle is precisely segmented again to obtain a new precise segmentation bounding rectangle, and its accuracy requirement is calculated. If the accuracy requirement is met, the cycle ends; otherwise, the cycle continues. In some embodiments, a deep convolutional neural network model can be used to precisely segment at least one element in the coarse segmentation. In some embodiments, historical medical images initially acquired before coarse segmentation can be used as training data, and a deep convolutional neural network model can be trained using historical precise segmentation result data. In some embodiments, historical medical images and historical precise segmentation result data are obtained from the medical scanning device 110. In some embodiments, historical medical images and historical precise segmentation result data of the scanned object can be obtained from the terminal 130, the processing device 140, and the storage device 150.

[0095] Sub-step 343b: If the judgment result is accurate, output the preliminary accurate segmentation result as the segmentation result.

[0096] In some embodiments, when the judgment result (i.e. the coarse segmentation result) is accurate, it can be determined that the positioning information of the element obtained by precise segmentation through the coarse segmentation result is accurate, and the preliminary precise segmentation result can be output.

[0097] In some embodiments, at least one element result data of the precise segmentation described above can be output. In some embodiments, to further reduce noise and optimize image display, image post-processing operations can be performed before the segmentation results are output. Image post-processing operations can include edge smoothing and / or image denoising. In some embodiments, edge smoothing can include smoothing or blurring to reduce noise or distortion in medical images. In some embodiments, smoothing or blurring can be performed using the following methods: mean filtering, median filtering, Gaussian filtering, and bilateral filtering.

[0098] Figure 11 These are exemplary comparison diagrams of segmentation results shown in some embodiments of this specification.

[0099] like Figure 11As shown, the top and bottom left images are cross-sectional and stereoscopic medical images obtained using traditional coarse segmentation techniques, respectively. The right images are cross-sectional and stereoscopic medical images obtained using the organ segmentation method provided in this application. A comparison reveals that the target organ segmentation result displayed on the right image is more complete than that displayed on the left image, reducing the risk of missing segmented organs, improving segmentation accuracy, and ultimately increasing overall segmentation efficiency.

[0100] Step 260: Register the first segmented image with the second segmented image to determine the spatial location of the third target structure set during the operation.

[0101] The third target structure set is the complete set of structures obtained after registering the first segmented image and the second segmented image. In some embodiments, the third target structure set may include the target organ (e.g., the target organ), blood vessels within the target organ, lesions, and other regions / organs (e.g., non-invasive regions, all vital organs). In some embodiments, in fast segmentation mode, other regions / organs may refer to non-invasive regions; in precise segmentation mode, other regions / organs may refer to all vital organs. In some embodiments, at least one element in the third target structure set is included in the first target structure set, and at least one element in the third target structure set is not included in the second target structure set. For example, when the first target structure set includes blood vessels within the target organ, the target organ, and lesions, and the second target structure set includes non-invasive regions (or all vital organs), the target organ, and lesions, the blood vessels within the target organ are included in the first target structure set but not in the second target structure set. In some embodiments, the fourth target structure set may also be considered as part of the third target structure set, for example, non-invasive regions and all vital organs outside the target organ.

[0102] In some embodiments, the first segmented image (i.e., the segmented image of the first target structure set obtained by segmenting the preoperative enhanced image) may include the precise structural features of the first target structure set (e.g., blood vessels in the preoperative target organ, the preoperative target organ, and the preoperative lesion); the second segmented image (i.e., the segmented image of the second target structure set obtained by segmenting the intraoperative scan image) may include the precise structural features of the second target structure set (e.g., the intraoperative target organ, the intraoperative lesion, and the intraoperative non-invasive area / all important organs). In some embodiments, before registration, the first segmented image and the second segmented image may be processed to separate the appearance features of the target structure set from the background. In some embodiments, the separation of appearance features from the background may employ artificial neural networks (linear decision functions, etc.), threshold-based segmentation methods, edge-based segmentation methods, image segmentation methods based on clustering analysis (e.g., K-means), or any other feasible algorithm, such as wavelet transform-based segmentation methods, etc.

[0103] The registration process is described below using an example where the first segmented image includes blood vessels and structural features of the preoperative target organ (e.g., the target organ) (i.e., the first target structure set includes blood vessels and the target organ itself), and the second segmented image includes the intraoperative target organ, intraoperative lesion, and structural features of the intraoperative non-invasive region / all important organs (i.e., the second target structure set includes the target organ, lesion, and non-invasive region / all important organs). It is understood that the structural features of the lesion are not limited to being included in the second segmented image. In other embodiments, the structural features of the lesion may also be included in the first segmented image, or the structural features of the lesion may be included in both the first and second segmented images.

[0104] Figure 12 This is an exemplary flowchart illustrating the registration process of a first segmented image and a second segmented image as shown in some embodiments of this specification. Step 261: Register the first segmented image and the second segmented image to determine the registration deformation field.

[0105] Registration is an image processing operation that uses spatial transformation to ensure that corresponding points in a first segmented image and a second segmented image are in the same spatial and anatomical position. The registration deformation field can be used to reflect the spatial positional changes between the first and second segmented images. In some embodiments, after registration, the intraoperative scan image can be transformed based on the registration deformation field to ensure that the transformed intraoperative scan image is completely consistent with the preoperative enhanced image in both spatial and anatomical position.

[0106] Figures 13 to 14 This is an exemplary flowchart illustrating the process of determining the registration deformation field in some embodiments of this specification. Figure 15These are exemplary demonstration diagrams showing the first segmented image and the second segmented image obtained after segmentation, as illustrated in some embodiments of this specification.

[0107] In some embodiments, step 261, which involves registering the first segmented image and the second segmented image to determine the registration deformation field, may include the following sub-steps: Sub-step 2611: Determine the first preliminary deformation field based on the registration between elements.

[0108] In some embodiments, the elements can be the outlines of elements in a first segmented image and a second segmented image (e.g., organ outlines, blood vessel outlines, lesion outlines). Registration between elements can refer to the registration between image regions covered by the element outlines (masks). For example... Figure 14 and Figure 15 The image region covered by the organ outline A of the target organ (such as the target organ) obtained after segmentation in the preoperative enhanced image (the area with the same or basically the same gray level in the dotted line area in the lower left figure), and the image region covered by the organ outline B of the target organ (such as the target organ) obtained after segmentation in the intraoperative scan image (the area with the same or basically the same gray level in the dotted line area in the lower right figure).

[0109] In some embodiments, a first preliminary deformation field (e.g., ) is obtained through region registration between the image region covered by organ contour A and the image region covered by organ contour B. Figure 14 The deformation field 1 in the figure. In some embodiments, the first preliminary deformation field can be a local deformation field. For example, the local deformation field about the liver contour is obtained by the preoperative liver contour A and the intraoperative contour B.

[0110] Sub-step 2612: Based on the first preliminary deformation field between elements, determine the second preliminary deformation field of the entire image.

[0111] A full-view image can be a region encompassing the element. For example, if the target organ is the liver, the full-view image could be an image of the entire abdominal cavity. Similarly, if the target organ is the lung, the full-view image could be an image of the entire thoracic cavity.

[0112] In some embodiments, a second preliminary deformation field for the entire image can be determined by interpolation based on a first preliminary deformation field. In some embodiments, this second preliminary deformation field can be a global deformation field. For example... Figure 14 The deformation field 2 is determined by interpolation of deformation field 1 to determine the overall size of the image.

[0113] Sub-step 2613: Based on the second preliminary deformation field of the whole image, deform the floating image to determine the registration map of the floating image.

[0114] The floating image can be an image to be registered, such as a preoperative enhanced image or an intraoperative scan image. For example, when registering an intraoperative scan image to a preoperative scan image, the floating image is the intraoperative scan image. The intraoperative scan image can be registered using a registration deformation field to ensure its spatial position matches that of the preoperative scan image. Similarly, when registering a preoperative enhanced image to an intraoperative scan image, the floating image is the preoperative enhanced image. The preoperative scan image can also be registered using a registration deformation field to ensure its spatial position matches that of the intraoperative scan image. The registration map of the floating image can be an image of an intermediate registration result obtained during the registration process. Taking the registration of a preoperative enhanced image to an intraoperative scan image as an example, the registration map of the floating image can be an intermediate intraoperative scan image obtained during the registration process. For ease of understanding, this specification uses the registration of a preoperative enhanced image to an intraoperative scan image as an example to describe the registration process in detail.

[0115] In some embodiments, such as Figure 14 As shown, based on the deformation field 2 of the acquired full image, the floating image, i.e., the preoperative enhanced image, is deformed to determine the registration map of the preoperative enhanced image, i.e., the intraoperative scan image of the intermediate registration result. For example, as... Figure 14 As shown, based on the deformation field of the peritoneal cavity where the liver is located, the preoperative enhanced image (peritoneal enhanced image) is deformed to obtain its registration map.

[0116] Sub-step 2614 involves registering the registration map of the floating image with the region of the first gray-level difference range in the reference image to obtain the third preliminary deformation field.

[0117] In some embodiments, the reference image refers to the target image before registration, or the unregistered target image. For example, when registering a preoperative enhanced image to an intraoperative scan image, the reference image refers to the intraoperative scan image before registration. In some embodiments, the third preliminary deformation field can be a local deformation field. In some embodiments, sub-step 2614 can be implemented as follows: performing pixel grayscale calculations on different regions of the registration map of the floating image and the reference image to obtain corresponding grayscale values; calculating the difference between the grayscale value of the registration map of the floating image and the grayscale value of the corresponding region of the reference image; when the difference is within a first grayscale difference range, elastically registering the registration map of the floating image and the corresponding region of the reference image to obtain the third preliminary deformation field. In some embodiments, the difference being within the first grayscale difference range can indicate that the difference between a region in the registration map of the floating image and the corresponding region in the reference image is not significant or relatively small. For example, if the first grayscale difference ranges from 0 to 150, and the grayscale difference between region Q1 in the registration map of the floating image and the same region in the reference image is 60, and the grayscale difference between region Q2 in the registration map of the floating image and the same region in the reference image is 180, then the difference between region Q1 in the two images (i.e., the registration map of the floating image and the reference image) is small, while the difference between region Q2 is large. Therefore, registration is performed only on region Q1 in the two images. In some embodiments, such as... Figure 14 As shown, elastic registration is performed on the registration map of the floating image and the region in the reference image that conforms to the first gray level difference range (the region with a relatively small difference) to obtain deformation field 3 (i.e. the third preliminary deformation field mentioned above).

[0118] Sub-step 2615: Based on the third preliminary deformation field, determine the fourth preliminary deformation field of the entire map.

[0119] In some embodiments, a fourth preliminary deformation field for the entire image is obtained by interpolation based on a third preliminary deformation field. In some embodiments, the fourth preliminary deformation field can be a global deformation field. In some embodiments, this step can be used to obtain a fourth preliminary deformation field about the global extent from a local third preliminary deformation field. For example, Figure 14 The deformation field 4 is determined by interpolation of the deformation field 3 to determine the overall size of the image.

[0120] Sub-step 2616: Based on the fourth preliminary deformation field, register the region of the second grayscale difference range to obtain the final registered map.

[0121] In some embodiments, the region of the second grayscale difference range can be the region where the grayscale value of the registration map of the floating image differs significantly from the grayscale value of the reference image. In some embodiments, a grayscale difference threshold (e.g., a grayscale difference threshold of 150) can be set. Regions where the difference between the grayscale value of the registration map of the floating image and the grayscale value of the reference image is less than the grayscale difference threshold are the regions of the first grayscale difference range, while those greater than the grayscale difference threshold belong to the regions of the second grayscale difference range.

[0122] In some embodiments, the final registration map can be obtained by subjecting the floating image (e.g., preoperative enhanced image) to multiple deformations based on at least one deformation field, resulting in an image with the same spatial and anatomical location as the intraoperative scan image. In some embodiments, such as Figure 6 As shown, based on the fourth preliminary deformation field, the region with the second gray-level difference range (i.e., the gray-level difference is relatively large) is registered to obtain the final registered map. For example, for the region outside the spleen with a relatively large gray-level difference, deformation is performed on this region using deformation field 4 to obtain the final registered map.

[0123] In some embodiments, utilizing Figures 13-14 The registration method described herein can map elements that are segmented in a floating image but not segmented in the reference image (e.g., blood vessels within the target organ) from the floating image to the reference image. Taking a preoperative enhanced image as an example, where the floating image is a preoperative enhanced image and the reference image is an intraoperative scan image, the blood vessels within the target organ are segmented in the preoperative enhanced image but not in the intraoperative scan image. Registration can then map the blood vessels within the target organ to the intraoperative scan image. It is understood that this method can also be used for registration of non-invasive regions in rapid segmentation mode and for registration of all important organs in precise segmentation mode. Figures 13-14 Similar results can be achieved through registration methods or simply by using the corresponding segmentation methods.

[0124] Step 262: Determine the spatial location of the corresponding element during surgery based on the registration deformation field and the spatial locations of at least some elements in the first target structure set in the preoperative enhanced image. In some embodiments, the spatial location of blood vessels (hereinafter referred to as blood vessels) within the target organ during surgery can be determined based on the registration deformation field and blood vessels within the target organ in the preoperative enhanced image.

[0125] In some embodiments, the spatial location of blood vessels during surgery can be determined based on the following formula (1), based on the registered deformation field and blood vessels in the preoperative enhanced image: (1) in, This indicates preoperative contrast-enhanced imaging. Represents the three-dimensional spatial coordinates of blood vessels. This represents the registration deformation field from preoperative enhanced images to intraoperative scan images. This indicates the spatial location of blood vessels in the surgical scan image. In some embodiments, It can also be understood as the offset of the three-dimensional coordinates of an element in the floating image (e.g., blood vessels in the target organ) to the three-dimensional coordinates in the final registered image.

[0126] Therefore, the blood vessels in the preoperative enhanced image can be deformed using the registration deformation field determined in step 261 to generate the same spatial position of the blood vessels during surgery.

[0127] In some embodiments, the lesion center point can be calculated based on the determined spatial location of blood vessels and lesions during surgery (including in a second segmented image of the intraoperative scan), and a safe zone around the lesion and a potential needle insertion area can be generated. In some embodiments, the safe zone around the lesion and the potential needle insertion area can be determined based on a determined invasive area and a non-invasive area. In some embodiments, a baseline path from the percutaneous needle insertion point to the lesion center point can be planned based on the potential needle insertion area and basic obstacle avoidance constraints. In some embodiments, basic obstacle avoidance constraints may include, but are not limited to, the needle insertion angle of the path, the needle insertion depth of the path, and the path not intersecting with blood vessels and vital organs.

[0128] Step 270: Plan the interventional path based on the spatial location of the third target structure set during the operation, and conduct a risk assessment based on the interventional path.

[0129] In some embodiments, the spatial locations of elements in the third target structure set (e.g., target organ, lesion, blood vessels within the target organ, non-invasive areas, all vital organs) can more comprehensively and accurately reflect the current condition of the scanned object (e.g., the patient). Interventional pathways can be planned based on the spatial locations of the third target structure set to enable surgical instruments (e.g., puncture needles) to effectively avoid blood vessels within the target organ, non-invasive areas, and / or all vital organs while successfully reaching the lesion, thereby reducing surgical risks.

[0130] In some embodiments, risk assessment based on the intervention path may include: determining the intervention risk value of at least some elements in the third target structure set; and conducting risk assessment based on the intervention risk value.

[0131] The intervention risk value can represent the degree of intervention risk of an element. In some embodiments, the higher the intervention risk value, the higher the degree of intervention risk, that is, the greater the intervention risk. For example, an element region with an intervention risk value of 8 points has a higher intervention risk than an element region with an intervention risk value of 6 points.

[0132] In some embodiments, the selection of elements in the third target structure set may be based on the pattern of interventional path planning. In some embodiments, the elements in the third target structure set used for risk assessment of the interventional path may differ depending on the pattern of interventional path planning. For example, in rapid planning mode, the elements in the third target structure set used for risk assessment of the interventional path may include blood vessels within the target organ and non-invasive areas. As another example, in precise planning mode, the elements in the third target structure set used for risk assessment of the interventional path may include blood vessels within the target organ and all vital organs.

[0133] In some embodiments, risk assessment based on the intervention path may further include: determining whether the intervention path passes through a preset element in the third target structure set, and when the determination result is yes, determining the intervention risk value of the preset risk object in the third target structure set.

[0134] In some embodiments, a preset element in the third target structure set may refer to a target organ. A preset risk object in the third target structure set may refer to blood vessels within the target organ. It is understood that preset risk objects may be included in at least some elements of the third target structure set used for risk assessment.

[0135] In some embodiments, when the interventional path crosses a target organ of a third target structure set, in the rapid planning mode, blood vessels and non-invasive areas within the target organ pose a certain risk relative to the interventional path, requiring the calculation of their interventional risk values ​​relative to the interventional path. In the precise planning mode, blood vessels within the target organ and external important organs / tissues also pose a certain risk relative to the interventional path, requiring the calculation of their interventional risk values ​​relative to the interventional path. In some embodiments, when the interventional path does not cross a target organ of the third target structure set, blood vessels within the target organ pose no risk relative to the interventional path, and the influence of blood vessels within the target organ on the interventional path need not be considered (it can also be considered that the interventional risk value of blood vessels within the target organ is zero). Therefore, when the interventional path does not cross a target organ of the third target structure set, in the rapid planning mode, only the interventional risk value of the non-invasive area relative to the interventional path needs to be calculated; in the precise planning mode, only the interventional risk value of external important organs / tissues relative to the interventional path needs to be calculated. By determining whether the interventional path crosses a target organ and by specifying the interventional risk values ​​of the elements to be calculated under different planning modes, a more reasonable risk assessment of the interventional path can be achieved.

[0136] In some embodiments, the method for determining whether the interventional path passes through the target organ can be: taking the intersection between the target organ mask and the interventional path; if the intersection is not an empty set, then the interventional path passes through the target organ; otherwise, it does not pass through the target organ.

[0137] Figure 16 This is an exemplary flowchart illustrating the determination of intervention risk values ​​for at least some elements in a third target structure set under a rapid planning mode as shown in some embodiments of this specification.

[0138] In some embodiments, the process 500 for determining the intervention risk values ​​of at least some elements in the third target structure set under the rapid planning mode may include the following steps: Step 510: Determine the risk level of the element based on the shortest distance between the element and the intervention path; Step 520: Determine the intervention risk value of the element based on the risk level.

[0139] In some embodiments, the elements in process 500 may include blood vessels and non-invasive regions within the target organ. Specifically, when the interventional path passes through target organs in the third set of target structures, the elements in process 500 may include blood vessels and non-invasive regions within the target organ. At the same distance, blood vessels and non-invasive regions within the target organ have different risk levels for the interventional path, and correspondingly different risk values. When the interventional path does not pass through target organs in the third set of target structures, the elements in process 500 may be non-invasive regions. Therefore, the risk level of the corresponding element can be determined based on the shortest distance between the blood vessel within the target organ and the interventional path, and the shortest distance between the non-invasive region and the interventional path, thereby determining the corresponding interventional risk value of the element.

[0140] As described above, the elements required for calculating the risk level and risk value differ depending on whether the interventional path crosses the target organ or not. The calculation methods for these elements are described below for each of the two scenarios: When the intervention path passes through the target organ, it is necessary to calculate the risk levels of the blood vessels and non-interventional areas in the target organ, as well as the intervention risk values. The calculation method for the intervention risk value of the blood vessels in the target organ is as follows: The shortest straight-line distance from the path to the blood vessel is L1. When 0 < L1 < M1, the risk level is the highest (denoted as the first risk level), and the corresponding intervention risk value is the first intervention risk value; when M1 < L1 < N1, the risk level is the second risk level, and the corresponding intervention risk value is the second intervention risk value; when N1 < L1 < P1, the risk level is the third risk level, and the corresponding intervention risk value is the third intervention risk value; when L1 > P1, the risk level and the intervention risk value are not considered. Among them, the first risk level is higher than the second risk level, and the second risk level is higher than the third risk level. For example, when M1 = 5mm, N1 = 10mm, and P1 = 15mm are taken, when 0 < L1 <= 5, the first intervention risk value corresponding to the first risk level can be 5 points; when 5 < L1 <= 10, the second intervention risk value corresponding to the second risk level is 3 points; when 10 < L1 <= 15, the third intervention risk value corresponding to the third risk level is 1 point; when L1 > 15, the risk level and the intervention risk value are not considered (it can also be understood that the intervention risk value is 0 points).

[0141] The calculation method for the intervention risk value of the non-interventional area is as follows: When the shortest straight-line distance from the path to the non-interventional area is L2, when 0 < L2 < A1, the risk level is the highest (denoted as the first risk level), and the corresponding intervention risk value is the first intervention risk value; when A1 < L2 < B1, the risk level is the second risk level, and the corresponding intervention risk value is the second intervention risk value; when B1 < L2 < C1, the risk level is the third risk level, and the corresponding intervention risk value is the third intervention risk value; when L2 > C1, the risk level and the intervention risk value are not considered. For example, when A1 = 3mm, B1 = 6mm, and C1 = 10mm are taken, when 0 < L2 <= 3, the first intervention risk value corresponding to the first risk level is 5 points; when 3 < L2 <= 6, the second intervention risk value corresponding to the second risk level is 3 points; when 6 < L2 <= 10, the third intervention risk value corresponding to the third risk level is 1 point; when L2 > 10, the risk level and the intervention risk value are not considered (it can also be understood that the intervention risk value is 0 points).

[0142] Among them, M1>A1, N1>B1, and P1>C1. This is because when blood vessels within the target organ are involved in the distance calculation, they are generally closer to the interventional path, thus the distance control between them is more stringent. Conversely, non-invasive areas are farther from the interventional path, so the distance control between them is more lenient. For example, when the distance between the blood vessels within the target organ and the interventional path, as well as the distance between the non-invasive area and the interventional path, is both 5mm, 5mm is more acceptable for the non-invasive area, resulting in an interventional risk score of 3. However, for the blood vessels within the target organ, the risk is higher, resulting in an interventional risk score of 5.

[0143] When the interventional path does not cross the target organ, only the risk level and interventional risk value of the non-invasive area need to be calculated. In this case, the calculation method for the risk level and interventional risk value of the non-invasive area is the same as that for the non-invasive area when it crosses the target organ, and will not be repeated here.

[0144] In some embodiments, the total risk value of at least one intervention path can be calculated; the intervention path with the minimum total risk value is determined as the optimal path, because the smaller the total risk value, the lower the risk. In some embodiments, the intervention risk values ​​of multiple intervention paths can be summed to obtain the total risk value. In some embodiments, the optimal path with the minimum total risk value is used to plan the intervention path.

[0145] In the rapid planning mode, since the rapid segmentation mode does not require segmenting all organs and tissues in the scene, it only needs to segment the non-invasive area. By extracting the location of blood vessels (and lesions) in the target organs that are not clearly visible in the intraoperative scan images through registration, when planning the interventional path, it is only necessary to bypass the non-invasive area so that the interventional path can directly reach the lesion, which helps to improve the efficiency of interventional planning and interventional surgery.

[0146] Figure 17 This is an exemplary flowchart illustrating the determination of intervention risk values ​​for at least some elements in a third target structure set under a precise planning mode as shown in some embodiments of this specification.

[0147] In some embodiments, the process 600 for determining the intervention risk value of at least some elements in the third target structure set under the precise planning mode may include the following steps: Step 610: Determine the risk level of the element based on the shortest distance between the element and the intervention path; Step 620: Determine the intervention risk value of the element based on the risk level; Step 630: Determine different priorities based on the preset rules of the elements, and set the corresponding preset weights for the intervention risk values.

[0148] In some embodiments, the elements in process 600 may include blood vessels within the target organ and all vital organs. Specifically, when the interventional path passes through target organs in the third set of target structures, the elements in process 600 may include blood vessels within the target organ and all vital organs. When the interventional path does not pass through target organs in the third set of target structures, the elements in process 600 may include all vital organs. Preset rules can be used to characterize the non-interventional importance of different elements to the planned interventional path. For example, in a planned interventional path, the non-interventional importance of blood vessels within the target organ and each vital organ among all vital organs may differ. Different elements have different priorities under preset rules. In some embodiments, the priority of each element can be determined according to the preset rules of the elements.

[0149] In some embodiments, in step 630, determining different priorities based on preset rules for elements and setting corresponding preset weights for interventional risk values ​​can be implemented as follows: Based on different priorities of elements, setting corresponding preset weights for interventional risk values. In some embodiments, different priorities of the segmented region (i.e., the element) can be determined according to the non-interventional importance of the segmented region. For example, segmented regions that are definitely non-interventional, such as blood vessels and important organs, are set to higher priorities. In some embodiments, different preset weights can be assigned to elements with different priorities. In some embodiments, the higher the priority, the larger the corresponding preset weight; the lower the priority, the smaller the corresponding preset weight. For example, the preset weight can be represented by W, where W∈{1, 0.8, 0.6}. When the priority is higher, a larger preset weight can be set (e.g., W=1); when the priority is lower, a smaller preset weight can be set (e.g., W=0.6).

[0150] When the intervention path passes through the target organ, it is necessary to calculate the risk levels of the blood vessels in the target organ and all important organs, as well as the intervention risk value. The calculation methods for the risk levels of the blood vessels in the target organ and the intervention risk value are the same as those in the fast mode, and will not be elaborated here. The calculation methods for the risk levels of important organs and the intervention risk value can be as follows: Based on the shortest distance between the intervention path and its adjacent important organs (i.e., the distance between the needle path and the point on the adjacent organ closest to the needle path), determine the risk level and intervention risk value of the divided organ area. In some embodiments, it can be determined whether the shortest distance between the needle path and its adjacent non-puncturable organ is within a set threshold range to determine the risk level and intervention risk value of the divided organ area. In some embodiments, the set threshold range can be determined by using a set of multiple constant thresholds, such as represented by X, Y, and Z, where X < Y < Z. The shortest distance between the intervention path and its adjacent non-puncturable organ can be represented as L3, and the intervention risk value can be represented as R. When it is determined that the intervention path passes through an organ, the planned intervention path immediately fails, and there is no need to evaluate the intervention risk value anymore; when 0 < L3 < X, the risk level is relatively high, and the set intervention risk value R is a; when X < L3 < Y, the risk level is the second, and the set intervention risk value is b; when Y < L3 < Z, the risk level is relatively low, and the set intervention risk value R is c; when L3 > Z, this risk can be ignored, and the set intervention risk value is 0, where a > b > c. In some embodiments, when the intervention path passes through the target organ, the corresponding priorities can be determined based on the preset rules of the blood vessels in the target organ and all important organs, and different weights are assigned to the intervention risk values of the blood vessels and all important organs with different priorities.

[0151] When the intervention path does not pass through the target organ, it is only necessary to calculate the risk levels of important organs and the intervention risk value. In this case, the calculation methods for the risk levels of important organs and the intervention risk value are the same as those when passing through the target organ, and will not be elaborated here. In some embodiments, when the intervention path does not pass through the target organ, the corresponding priorities can be determined based on the preset rules of all important organs, and different weights are assigned to the intervention risk values of all important organs with different priorities.

[0152] In some embodiments, based on the intervention risk value, the planned intervention path can be implemented as follows: Calculate the weighted risk value of at least one intervention path; Determine the intervention path with the smallest weighted risk value as the optimal path. In some embodiments, the weighted risk value can be obtained by weighted calculation of the intervention risk values of multiple intervention paths. In some embodiments, the optimal path with the smallest weighted risk value is used to plan the intervention path. In some embodiments, the weighted risk value can be represented by F, and the weighted risk value F can be calculated through the following calculation formula (2): (2) The smaller the weighted risk value F, the farther the needle path is from important organs and blood vessels, and the lower the risk.

[0153] In the precision planning mode, when planning interventional (such as puncture needle) pathways, the outlines of the target organ and other tissues in the scene can be used to set interventional (such as puncture) priorities based on the interventional risk value. A reasonable path can be planned from the intervention point (such as the needle insertion point) to the lesion area, which can avoid high-priority non-interventional areas (such as blood vessels, important organs, etc.) and obtain potential needle insertion space, thereby improving the efficiency of interventional planning and interventional surgery.

[0154] It should be noted that, under the above two modes (i.e., rapid planning mode and precise planning mode), the value of the intervention risk corresponding to different risk levels can be set according to the actual situation, and this manual does not impose further restrictions on this.

[0155] Figure 18 This is an exemplary flowchart of the image anomaly detection process 700 shown in some embodiments of this specification.

[0156] In some embodiments, the image anomaly detection process 700 may include the following steps: Step 710: Acquire intraoperative scan images; Step 720: Detect image abnormalities in the intraoperative scan images; Step 730: Based on the detected image anomalies, determine the corresponding image anomaly type; Step 740: Based on the image anomaly type, determine whether to perform quantitative calculation; Step 750: Determine the degree of image anomaly based on the judgment result of whether quantitative calculation is performed.

[0157] In some embodiments, the method of acquiring intraoperative scan images may be found in [reference needed]. Figures 2 to 3 The relevant descriptions will not be repeated here. In some embodiments, image anomalies may include non-compliant portions of the image data containing complications. In some embodiments, complications may include bleeding, pneumothorax, effusion, etc.

[0158] In some embodiments, generative adversarial networks (GANs) modeled using deep learning can be used to detect image anomalies. Anomalies can be detected by comparing the modeled data with normal data. In some embodiments, at least one of the following methods can be used to detect image anomalies: thresholding, image segmentation, etc. In some embodiments, thresholding can be implemented as follows: since different complications have inconsistent feedback in images—pneumothorax, hemorrhage, effusion, etc.—the pixel value distribution ranges in images differ. A pixel threshold can be set to distinguish which complication the pixel values ​​in the abnormal area belong to. In some embodiments, image segmentation can be implemented as follows: after acquiring image anomalies, deep learning is used to segment the anomalies, classifying the pixels in the abnormal pixel area to determine which complication it belongs to. If it is not a complication, the surgical procedure can continue; otherwise, complications such as hemorrhage, effusion, and pneumothorax can be quickly identified and judged.

[0159] In some embodiments, the surgical procedure proceeds differently depending on the type of image abnormality. For example, if the image abnormality is pneumothorax, an alarm can be issued to the operator, and the surgical procedure can be terminated. As another example, if the image abnormality is bleeding or effusion, the amount of bleeding or effusion can be quantitatively calculated, and the surgical procedure can be continued or terminated based on the result of the quantitative calculation. In some embodiments, the corresponding amount of bleeding or effusion can be calculated based on the percentage of the image area. In some embodiments, it can be determined whether the amount of bleeding or effusion exceeds a preset threshold (e.g., a preset blood volume threshold, a preset fluid volume threshold). If it does not exceed the threshold, a small amount of bleeding or effusion will not affect the progress of the interventional procedure, and the procedure can continue with continuous observation. If the amount of bleeding or effusion exceeds the preset threshold, a safety issue will arise, and a warning message can be issued to the doctor.

[0160] In some embodiments, the quantitative calculation results determine the degree of image abnormality. For example, if the amount of bleeding or effusion exceeds a preset threshold, the degree of image abnormality can be determined to be high; if the amount of bleeding or effusion does not exceed the preset threshold, the degree of image abnormality can be determined to be low.

[0161] In some embodiments, alarm prompts can be issued based on the type and severity of the image abnormality. For example, if the image abnormality is pneumothorax, a prompt message can be sent to the operator to stop the intervention. In some embodiments, if the image abnormality is bleeding or effusion, different alarm prompts can be issued based on the severity of the bleeding or effusion. For example, if the severity of the image abnormality is high, a prompt message can be sent to the operator to stop the intervention. Conversely, if the severity of the image abnormality is low, a prompt message can be sent to the operator indicating that the intervention can proceed and that continuous observation should be conducted.

[0162] Image anomaly detection can effectively detect complications that may occur during and after the procedure, preventing dangerous complications from happening. Even if complications occur, it can promptly alert the doctor, allowing the procedure to be stopped in time for appropriate treatment, thus improving the safety of interventional surgery.

[0163] Figure 19 This is an exemplary flowchart of the postoperative assessment process shown in some embodiments of this specification.

[0164] In some embodiments, the postoperative evaluation process 800 may include the following steps: Step 810: Register the planned interventional path and the actual interventional path to the intraoperative scan images; Step 820: Determine the deviation between the actual intervention path and the planned intervention path; Step 830: Determine whether the deviation intersects with a specific element in the third target structure set of the intraoperative scan image; Step 840: Determine the corresponding postoperative feedback information based on the judgment result.

[0165] In some embodiments, the planned interventional path can be based on preoperative contrast-enhanced images and intraoperative scans; the actual interventional path can be based on postoperative scans. In some embodiments, postoperative scans refer to images obtained by medical scanning equipment after the interventional procedure (e.g., a patient). For information on how postoperative scans are acquired, please refer to the preceding text (e.g., ...). Figure 2 The relevant descriptions will not be repeated here. Specific elements in the third target structure set of intraoperative scan images can be the fourth target structure set. That is, in rapid planning mode, specific elements refer to non-invasive areas; in precise planning mode, specific elements refer to all external important organs / tissues.

[0166] In some embodiments, the planned interventional path and the actual interventional path can be registered to the intraoperative scan image, and registration calculations can be performed to obtain the registration deformation field. In some embodiments, the registered interventional path can be displayed, and the difference between the registered actual interventional path and the planned interventional path can be calculated. If there is a deviation between the actual interventional path and the planned interventional path, the deviation is extracted, and it is determined whether the deviation intersects with the non-interventional area or all important organs / tissues in the intraoperative scan image. If the intersection is not null, it proves that the actual interventional path may pass through the non-interventional area or all important organs / tissues, which may affect solid organs. In this case, the corresponding postoperative feedback information can be determined to be a reminder to the clinician; if the intersection is null, the corresponding postoperative feedback information is determined to be no reminder; if there is no difference between the actual interventional path and the planned interventional path, the corresponding postoperative feedback information is determined to be no reminder.

[0167] Figure 20 This is an exemplary flowchart of the postoperative assessment process shown in some embodiments of this specification.

[0168] In some embodiments, after acquiring preoperative enhanced images and postoperative scan images, the lesion (original lesion) and its surrounding organ region are segmented, and a region of interest (ROI) is extracted from this region for registration, so that the positions of the preoperative lesion and the postoperative original lesion region correspond. These are then merged and displayed to facilitate the doctor's analysis of the surgical results. In some embodiments, after obtaining the segmentation results of the lesion and the original lesion region, the region is extracted based on the segmentation results. On the one hand, the area of ​​the lesion that changed postoperatively can be calculated to evaluate the efficacy of the surgery; on the other hand, the pixels in this region can be analyzed to determine whether any lesions still exist and the area range of the lesions. In some embodiments, based on the postoperative scan images, image anomaly detection, i.e., postoperative complication detection and identification operations, can be performed. For details, please refer to [link to relevant documentation]. Figure 18 The relevant descriptions will not be repeated here. In some embodiments, based on postoperative scan images, the path during the actual intervention process can be extracted using methods such as overthreshold segmentation and deep learning. This path is then registered with the planned intervention path (i.e., needle path comparison), and it is determined whether there are any changes to assess the impact of these changes and achieve accurate evaluation. Specifically, when there is a deviation between the actual intervention path and the planned intervention path, the deviation is determined, and it is determined whether this deviation intersects with a specific element in the third target structure set (i.e., the fourth target structure set) of the intraoperative scan images. If the intersection is an empty set, no alert is issued; if the intersection is not an empty set, it indicates that the path passes through the fourth target structure set, and an alert can be issued to the clinician. When there is no deviation between the actual intervention path and the planned intervention path, no alert is issued.

[0169] It should be noted that the above descriptions of processes 200, 500, 600, 700, and 800 are for illustrative purposes only and do not limit the scope of this specification. Those skilled in the art can make various modifications and changes to processes 200, 500, 600, 700, and 800 under the guidance of this specification.

[0170] Figure 21 This is an exemplary structural block diagram of a medical image processing system 900 for interventional surgery, as shown in some embodiments of this specification.

[0171] The interventional surgical medical image processing system 900 may include a first acquisition module 910, a second acquisition module 920, a first segmentation module 930, a third acquisition module 940, a second segmentation module 950, a deformation registration module 960, and a risk assessment module 970. In some embodiments, the first acquisition module 910, the second acquisition module 920, the first segmentation module 930, the third acquisition module 940, the second segmentation module 950, the deformation registration module 960, and the risk assessment module 970 may be configured to... Figure 1 This is implemented in the medical image processing system 100 shown, such as in the medical scanning device 110.

[0172] The first acquisition module 910 is used to acquire the pattern of the planned interventional path; the second acquisition module 920 is used to acquire preoperative enhanced images; the first segmentation module 930 is used to segment the first target structure set of the preoperative enhanced images to obtain the first segmented image of the first target structure set; the third acquisition module 940 is used to acquire intraoperative scan images; the second segmentation module 950 is used to segment the second target structure set of the intraoperative scan images to obtain the second segmented image of the second target structure set; the deformation registration module 960 is used to register the first segmented image and the second segmented image to determine the spatial position of the third target structure set during the operation, and the element selection of the third target structure set is based on the pattern of the planned interventional path; the risk assessment module 970 is used to plan the interventional path based on the spatial position of the third target structure set during the operation and to perform risk assessment based on the interventional path.

[0173] It should be noted that for more technical details regarding the execution of corresponding procedures or functions by the first acquisition module 910, the second acquisition module 920, the first segmentation module 930, the third acquisition module 940, the second segmentation module 950, the deformation registration module 960, and the risk assessment module 970 to achieve surgical image assistance, please refer to [link to relevant documentation]. Figures 1 to 20 The details of the medical image processing method for interventional surgery described in any of the embodiments shown will not be repeated here.

[0174] The above description of the interventional surgical medical image processing system 900 is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art will recognize that various forms and details of improvements and changes can be made to the application of the described methods and systems without departing from the principles of this specification. However, such changes and modifications will not depart from the scope of this specification. In some embodiments, the interventional surgical medical image processing system 900 may include one or more other modules. For example, the interventional surgical medical image processing system 900 may include a storage module for storing data generated by the modules of the interventional surgical medical image processing system 900.

[0175] Some embodiments of this specification also provide an interventional surgical medical image processing apparatus, including a processor for executing the interventional surgical medical image processing method described in any embodiment. See details... Figures 1 to 20 The relevant descriptions will not be repeated here.

[0176] Some embodiments of this specification also provide a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions, the computer executes the interventional surgical medical image processing method as described in any of the above embodiments. See details below. Figures 1 to 20 The relevant descriptions will not be repeated here.

[0177] The surgical image-assisted methods, systems, devices, and computer-readable storage media provided in the embodiments of this specification have at least the following beneficial effects: (1) First, taking into account the good imaging effect of preoperative enhanced images on targets such as blood vessels and lesions, and the advantage of intraoperative scan images that are close to the patient's real condition, a coarse-to-fine optimization segmentation method combined with deep learning is adopted in the segmentation process. Precise organ localization provides support for precise organ segmentation, thereby improving segmentation efficiency and image processing robustness. (2) Secondly, by using the soft connected domain analysis method in the coarse segmentation stage, the target structure set region is accurately preserved while the false positive region is effectively eliminated. This firstly improves the accuracy of element localization in the coarse localization stage and directly helps to extract the bounding box of element mask localization information in the subsequent stage, thereby improving the segmentation efficiency. (3) Furthermore, in response to the unfavorable situation that the coarse positioning is inaccurate but not ineffective in the coarse segmentation stage, the adaptive sliding window calculation and corresponding sliding window operation can be used to fill in the missing parts of the positioning area and can automatically plan and execute reasonable sliding window operations, thereby reducing the dependence of the fine segmentation stage on the coarse positioning results and improving the segmentation accuracy without significantly increasing the segmentation time and computing resources. (4) Furthermore, even when coarse positioning fails, the element mask can be accurately positioned based on the preset element positioning coordinates, which not only improves the segmentation accuracy but also reduces the segmentation time and the amount of segmentation calculation, further improving the segmentation efficiency. (5) Then, because the overall workflow of the target structure set segmentation fully considers various adverse situations that reduce the accuracy of target structure segmentation, it is applicable to the effective implementation of different types of target structure set segmentation tasks, with high segmentation accuracy and segmentation robustness. (6) Furthermore, by combining the advantages of preoperative enhanced images and intraoperative scan images, a fast segmentation mode and a precise segmentation mode are set up (the planning mode is only for intraoperative scan images). Different path planning schemes are determined according to the selected planning mode. In the fast segmentation mode, the planning speed is fast and the time is short; in the precise mode, there are more choices of planning paths and stronger robustness. While providing strong processing applicability, it can also ensure system stability and interventional safety, so that the preoperative planning can achieve a high degree of accuracy, so as to better assist the accurate implementation of the corresponding puncture path during the operation and obtain a more ideal surgical effect. (7) Furthermore, two fully automatic modes are provided for interventional surgery planning. In the precise planning mode, when planning the interventional (such as puncture needle) path, a reasonable path can be planned to avoid high-priority blood vessels and important organs, and potential needle insertion space can be obtained, thereby improving the efficiency of interventional planning and interventional surgery. In the rapid planning mode, when planning the interventional path, the rapid segmentation mode only needs to bypass the non-interventional area, so that the interventional path can directly reach the lesion, which also helps to improve the efficiency of interventional planning and interventional surgery. (8) In addition, it can efficiently and accurately plan the best intervention path automatically and analyze the risks of the intervention path, providing good preoperative planning guidance for interventional surgery. During the operation, it provides real-time detection and identification of complications, further improving the safety of the intervention process. In addition, the workflow also realizes the postoperative evaluation function, which can assist the operator to accurately evaluate the surgical process and surgical results, improving the efficiency and safety of the operation.

[0178] The basic concepts have been described above. Obviously, for those skilled in the art, the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0179] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0180] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments of the invention that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on existing servers or mobile devices.

[0181] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0182] In some embodiments, numbers describing the quantity of components and attributes are used. It should be understood that such numbers used in the description of embodiments are modified in some examples with the terms "approximately," "approximately," or "generally." Unless otherwise stated, "approximately," "approximately," or "generally" indicates that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may be changed depending on the characteristics required by individual embodiments. In some embodiments, numerical parameters should take into account specified significant digits and employ a general method of digit reservation. Although the numerical ranges and parameters used to confirm their breadth of range in some embodiments of this specification are approximate values, in specific embodiments, such values ​​are set as precisely as feasible.

[0183] Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments described herein. Other variations may also fall within the scope of this specification. Therefore, alternative configurations of the embodiments described herein are intended to be illustrative rather than limiting, and should be considered consistent with the teachings of this specification. Accordingly, the embodiments described herein are not limited to those explicitly introduced and described herein.

Claims

1. A method for processing medical images in interventional surgery, the method comprising: Patterns for obtaining planning intervention paths; Obtain preoperative contrast-enhanced images; The first target structure set of the preoperative enhanced image is segmented to obtain a first segmented image of the first target structure set; Acquire intraoperative scan images; The second target structure set in the surgical scan image is segmented to obtain a second segmented image of the second target structure set; The first segmented image and the second segmented image are registered to determine the spatial location of the third target structure set during surgery. The elements of the third target structure set are selected based on the planned interventional path pattern. The planned interventional path pattern includes at least a rapid planning pattern or a precise planning pattern. The intervention path is planned based on the spatial location of the third target structure set during the surgery.

2. The method according to claim 1, further comprising: Risk assessment is conducted based on the aforementioned intervention path; Wherein, at least one element in the third target structure set is included in the first target structure set, and at least one element in the third target structure set is not included in the second target structure set.

3. The method according to claim 2, wherein the risk assessment based on the intervention path includes: Determine the intervention risk value of at least some elements in the third target structure set; A risk assessment is conducted based on the aforementioned intervention risk value.

4. The method according to claim 3, wherein the risk assessment based on the intervention path includes: Determine whether the intervention path passes through a preset element in the third target structure set. If the determination result is yes, determine the intervention risk value of the preset risk object in the third target structure set.

5. The method according to any one of claims 2 to 4, wherein determining the intervention risk value of at least some elements in the third target structure set comprises: The risk level of the element is determined based on the shortest distance between the element and the intervention path; The intervention risk value of the element is determined based on the risk level.

6. The method according to any one of claims 2 to 4, wherein determining the intervention risk value of at least some elements in the third target structure set comprises: The risk level of the element is determined based on the shortest distance between the element and the intervention path; The intervention risk value of the element is determined based on the risk level. Different priorities are determined based on the preset rules of the elements, and corresponding preset weights are set for the intervention risk values.

7. The method according to claim 1, wherein the ratio of the total volume of the elements of the third target structure set in the rapid planning mode to the total volume of the elements of the third target structure set in the precise planning mode is greater than a preset efficiency factor m.

8. The method according to claim 1, further comprising: Acquire intraoperative scan images; The scanned images during the surgery were examined for abnormalities. Based on the detected image anomalies, determine the corresponding image anomaly type; Based on the image anomaly type, determine whether to perform quantitative calculation; The degree of image anomaly is determined based on the results of whether or not quantitative calculations are performed.

9. The method according to claim 8, further comprising: The planned interventional path obtained based on the preoperative enhanced image and the intraoperative scan image, and the actual interventional path obtained based on the postoperative scan image are registered to the intraoperative scan image. It is then determined whether the deviation between the actual interventional path and the planned interventional path intersects with a specific element in the third target structure set of the intraoperative scan image. Based on the determination result, the corresponding postoperative feedback information is determined.

10. A medical image processing system for interventional surgery, comprising: The first acquisition module is used to acquire the pattern of the planning intervention path; The second acquisition module is used to acquire preoperative enhanced images; The first segmentation module is used to segment the first target structure set of the preoperative enhanced image to obtain the first segmented image of the first target structure set. The third acquisition module is used to acquire intraoperative scan images; The second segmentation module is used to segment the second target structure set of the surgical scan image to obtain a second segmented image of the second target structure set. A deformation registration module is used to register the first segmented image and the second segmented image to determine the spatial location of a third target structure set during surgery. The elements of the third target structure set are selected based on the planned interventional path mode. The planned interventional path mode includes at least a rapid planning mode or a precise planning mode. The risk assessment module is used to plan the intervention path based on the spatial location of the third target structure set during the surgery.

Images