Image display method, system and device and storage medium

Abstract

The invention discloses an image display method, system and device and a storage medium, and the method comprises the steps: correcting a second medical image obtained in an operation preparation stage through a first medical image collected in a preoperative stage, obtaining a third medical image of a specified focus part in the operation preparation stage, and displaying the third medical image. And in response to a selection operation performed on the third medical image by the user, determining an initial operation image region for the to-be-processed bone tissue in the third medical image, further adjusting the initial operation image region according to an image region where a nervous tissue in the soft tissue in the third medical image is located, and determining and displaying an actual operative image region. The second medical image is corrected through the first medical image, and then the initial operation image area is adjusted through the image area where the nervous tissue in the soft tissue in the obtained third medical image is located, so that the actual operation image area which is used for the bone grinding operation process and can avoid damage to the nervous tissue is obtained.

Description

Technical Field

[0001] This specification relates to the field of surgical procedures, and more particularly to a method, system, apparatus, and storage medium for displaying images. Background Technology

[0002] Currently, bone-shaping surgery has a wide range of applications. For example, in the case of the spinal bones, bone-shaping surgery can be used to treat the spinal bones, thereby reducing the compression of nerve roots or the spinal cord. However, bone-shaping techniques require a high degree of precision, and the surgical risks are relatively high.

[0003] Current bone-shaping surgeries primarily rely on manual manipulation by medical personnel. However, during the procedure, the surgeon's field of vision is often limited, preventing direct observation of the bone-shaping area and increasing the risk of accidental damage to other tissues. For example, in endoscopic spinal bone-shaping surgery, the surgeon's field of vision is extremely limited by the spinal endoscope. Bone tissue obstructs this view, making it impossible to directly observe the size, depth, and surrounding vital tissues of the bone-shaping area. Particularly concerning are the nerve tissues surrounding the bone; even with strong surgical stability, there is a risk of accidental injury to important nerves near the bone. To minimize this risk, the area and amount of bone removed must be reduced, significantly diminishing the effectiveness of the procedure. Therefore, current manual bone-shaping surgeries struggle to achieve a high degree of surgical completion while preventing nerve damage.

[0004] Therefore, how to avoid nerve damage during bone-shaping surgery is an urgent problem to be solved. Summary of the Invention

[0005] This specification provides a method, system, apparatus, and storage medium for displaying images, in order to partially solve the aforementioned problems existing in the prior art.

[0006] The following technical solution is adopted in this specification:

[0007] This specification provides an image display method, including:

[0008] Acquire a first medical image of a designated lesion site of a patient, wherein the first medical image is obtained by acquiring the bone tissue to be treated and the soft tissue surrounding the bone tissue at the designated lesion site using a high-precision image acquisition device during the preoperative stage.

[0009] While the patient is in the surgical preparation stage, an image of the designated lesion site is obtained based on a pre-set scanning device used in the surgical preparation stage, which serves as a second medical image. The accuracy of the positional relationship between the bone tissue to be treated and the soft tissue represented by the second medical image is lower than that of the first medical image.

[0010] Using the first medical image, the positional relationship between the bone tissue to be processed and the soft tissue in the second medical image is corrected to obtain and display a third medical image of the designated lesion site when the patient is in the surgical preparation stage;

[0011] In response to a selection operation performed by the user on the third medical image, an initial surgical image region for the bone tissue to be processed is determined in the third medical image;

[0012] The initial surgical image region is adjusted based on the image region where the nerve tissue in the soft tissue is located in the third medical image, so as to determine and display the actual operable image region in the third medical image.

[0013] Optionally, the initial surgical image region is adjusted based on the image region where the nerve tissue in the soft tissue is located in the third medical image to determine the actual operable image region in the third medical image, specifically including:

[0014] The image region containing the neural tissue is identified from the initial surgical image region and designated as the neural image region;

[0015] The neural image region is removed from the initial surgical image region, and the remaining image region in the initial surgical image region is determined as the actual operable image region.

[0016] Optionally, the initial surgical image region is adjusted based on the image region where the nerve tissue in the soft tissue is located in the third medical image to determine the actual operable image region in the third medical image, specifically including:

[0017] The image region containing the neural tissue located outside the initial surgical image region is identified from the third medical image and designated as the neural image region;

[0018] Based on the neural image region, the surgical avoidance image region is determined;

[0019] Based on the surgical avoidance image region, the initial surgical image region is adjusted to determine the actual operable image region.

[0020] Optionally, a surgical avoidance image region is determined based on the neural image region, specifically including:

[0021] The surgical avoidance image region is divided into areas with the center point of the neural image region as the center and a preset safe distance threshold as the radius.

[0022] Optionally, the initial surgical image region is adjusted based on the surgical avoidance image region to determine the actual operable image region, specifically including:

[0023] Determine the intersection region of the surgical avoidance image region and the initial surgical image region;

[0024] The intersection region is removed from the initial surgical image region, and the remaining image region in the initial surgical image region is determined as the actual operable image region.

[0025] Optionally, the method further includes:

[0026] Based on the actual position of the surgical tool, mark the area of ​​action of the surgical tool in the designated lesion site within the actual operable image area.

[0027] This specification provides a surgical system, which includes: a scanning device, a display device, and a surgical robot;

[0028] The scanning device is used to scan a designated lesion site of the patient during the surgical preparation stage, obtain an image of the designated lesion site as a second medical image, and send the second medical image to the display device;

[0029] The display device is configured to receive the second medical image sent by the scanning device, and acquire a high-precision image obtained by a high-precision image acquisition device during the preoperative stage of acquiring the bone tissue to be treated and the soft tissue surrounding the bone tissue at the designated lesion site, as the first medical image. Using the first medical image, the positional relationship between the bone tissue to be treated and the soft tissue in the second medical image is corrected to obtain and display a third medical image of the designated lesion site when the patient is in the surgical preparation stage. In response to a user's selection operation on the third medical image, an initial surgical image region for the bone tissue to be treated is determined in the third medical image. Based on the image region where the nerve tissue in the soft tissue is located in the third medical image, the initial surgical image region is adjusted to determine and display an actual operable image region in the third medical image. Surgical instructions are generated based on the actual operable image region and sent to the surgical robot. The accuracy of the positional relationship between the bone tissue to be treated and the soft tissue represented by the first medical image is higher than that of the second medical image.

[0030] The surgical robot is used to receive the surgical instructions sent by the display device and control the surgical tools to perform surgery on the designated lesion site according to the surgical instructions.

[0031] Optionally, the display device is configured to identify the image region where the nerve tissue is located from the initial surgical image region as the nerve image region, remove the nerve image region from the initial surgical image region, and determine the remaining image region in the initial surgical image region as the actual operable image region.

[0032] Optionally, the display device is configured to identify, from the third medical image, an image region containing nerve tissue located outside the initial surgical image region, as a nerve image region; determine a surgical avoidance image region based on the nerve image region; and adjust the initial surgical image region based on the surgical avoidance image region to determine the actual operable image region.

[0033] Optionally, the display device is used to divide the surgical avoidance image region with the center point of the neural image region as the center and a preset safe distance threshold as the radius.

[0034] Optionally, the display device is used to determine the intersection area of ​​the surgical avoidance image area and the initial surgical image area, remove the intersection area from the initial surgical image area, and determine the remaining image area in the initial surgical image area as the actual operable image area.

[0035] Optionally, the system further includes: an optical tracker and an optical tracer;

[0036] The display device is used to mark the area of ​​action of the surgical tool in the specified lesion site in the actual operable image area, based on the actual position of the surgical tool determined by the optical tracker installed on the surgical robot using the optical tracer.

[0037] Optionally, the display device is used during surgery to mark the image areas where bone reshaping has been completed and the image areas where bone reshaping has not yet been performed in the actual operable image area using different colors in a specified display manner.

[0038] This specification provides an image display device, including:

[0039] The acquisition module is used to acquire a first medical image of a specified lesion site of a patient, wherein the first medical image is obtained by acquiring the bone tissue to be treated and the soft tissue around the bone tissue to be treated at the specified lesion site through a high-precision image acquisition device in the preoperative stage.

[0040] The scanning module is used to obtain an image of the designated lesion site based on a preset scanning device used in the surgical preparation stage when the patient is in the surgical preparation stage, as a second medical image. The accuracy of the positional relationship between the bone tissue to be processed and the soft tissue represented by the second medical image is lower than that of the first medical image.

[0041] The display module is used to correct the positional relationship between the bone tissue to be processed and the soft tissue in the second medical image based on the first medical image, and to obtain and display a third medical image of the designated lesion site when the patient is in the surgical preparation stage.

[0042] The first determining module is configured to determine an initial surgical image region for the bone tissue to be processed in the third medical image in response to a selection operation performed by the user on the third medical image.

[0043] The second determining module is used to adjust the initial surgical image region based on the image region where the nerve tissue in the soft tissue is located in the third medical image, so as to determine and display the actual operable image region in the third medical image.

[0044] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0045] In the image display method provided in this specification, a first medical image of the patient's designated lesion site is first acquired by acquiring the bone tissue to be treated and the soft tissue surrounding the bone tissue at the designated lesion site using a high-precision image acquisition device during the preoperative stage. Then, during the surgical preparation stage, an image of the designated lesion site with a lower accuracy in representing the positional relationship between the bone tissue to be treated and the soft tissue than the first medical image is obtained based on a preset scanning device used in the surgical preparation stage. This image is then used as a second medical image. The positional relationship between the bone tissue to be treated and the soft tissue in the second medical image is then corrected using the first medical image to obtain and display a third medical image of the designated lesion site of the patient during the surgical preparation stage. In response to the user performing a selection operation on the third medical image, an initial surgical image region for the bone tissue to be treated is determined in the third medical image. Then, based on the image region where the nerve tissue in the soft tissue is located in the third medical image, the initial surgical image region is adjusted to determine and display the actual operable image region in the third medical image.

[0046] As can be seen from the above method, the present invention can correct the second medical image by using the first medical image acquired in the preoperative stage to obtain a third medical image of the lesion site of the patient in the surgical preparation stage. The accuracy of the positional relationship between the bone tissue and soft tissue to be processed represented by this third medical image is higher than that of the second medical image. Then, the image area selected by the user can be received from the third medical image as the initial surgical image area. Then, the initial surgical image area can be adjusted by the image area where the nerve tissue in the soft tissue is located in the determined third medical image to obtain the actual operable image area for bone grinding surgery that can avoid damage to the nerve tissue, and the actual operable image area is displayed. Attached Figure Description

[0047] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0048] Figure 1 This is a flowchart illustrating an image display method provided in this specification;

[0049] Figure 2 This is a schematic diagram of an image provided in this specification that can characterize the positional relationship between the bone tissue to be treated and the soft tissue surrounding the bone tissue;

[0050] Figure 3 This is a schematic diagram of an image representing the positional relationship between the vertebra to be treated and the soft tissues surrounding the vertebra, as provided in this specification.

[0051] Figure 4 This is a schematic diagram of a second medical image obtained from a scanning device, as provided in this specification.

[0052] Figure 5 This is a schematic diagram of a surgical system provided in this specification;

[0053] Figure 6 This is a schematic diagram illustrating the progress of bone reshaping in an actual operable image area, as provided in this specification.

[0054] Figure 7 This is a schematic diagram of the structure of an image display device provided in this specification. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0056] The technical solutions provided by the various embodiments of this specification are described in detail below with reference to the accompanying drawings. In the existing field of surgery, bone reshaping surgery is often performed manually by medical personnel based on their personal experience. However, performing bone reshaping surgery manually often carries a high surgical risk because soft tissue containing nerve tissue often surrounds bone tissue. Medical personnel cannot directly observe the bone reshaping area during surgery. For example, in endoscopic surgery of the spine, medical personnel can only observe the bone tissue and surrounding nerve tissue through the endoscopic view provided by the spinal endoscope. It is difficult to accurately observe the location of the nerve tissue around the patient's bone tissue in real time during the surgery. This makes it easy to accidentally injure the patient's nerve tissue during bone reshaping surgery. For areas with less nerve tissue, such as the mandibular angle, the surgical risk of bone reshaping surgery in these areas is not high. However, for areas with more nerve tissue, such as the spine, the surgical risk is relatively high, and accidental injury to the nerve tissue in the spine is unacceptable.

[0057] Based on the above issues, medical staff often adopt a more conservative approach when performing high-risk bone-shaping surgeries, such as reducing the area of ​​bone removal to avoid damaging nerve tissue. This approach reduces the possibility of nerve damage, but conversely, reducing the area of ​​bone removal also reduces the success rate of the surgery. A less successful bone-shaping surgery cannot enable patients to achieve a high level of recovery.

[0058] As mentioned above, the existing methods of performing bone-shaping surgery have significant drawbacks. Because medical personnel have difficulty observing the soft tissues surrounding the bone to be treated during surgery, it is easy to accidentally injure the patient's nerve tissue during the bone-shaping process. To avoid injuring nerve tissue, the area of ​​bone-shaping needs to be reduced, which results in a low success rate of bone-shaping surgery and an inability to treat patients effectively.

[0059] Based on this, this specification provides an image display method. By correcting a second medical image obtained during the surgical preparation stage using a first medical image acquired before surgery, a third medical image is obtained that can be used to select the initial surgical image region. Then, the initial surgical region is adjusted according to the image region where the nerve tissue in the soft tissue is located in the third medical image that can intuitively show the patient's lesion site, so as to obtain the actual operable region for the bone-shaping surgery. By performing the surgery through the actual operable region, the completion of the bone-shaping surgery can be ensured while avoiding damage to the nerve tissue around the bone tissue.

[0060] The solution provided in this manual will be described in detail below.

[0061] Figure 1 This specification provides a flowchart illustrating an image display method, including:

[0062] S101: Acquire the first medical image of the specified lesion site on the patient.

[0063] For ease of explanation, the methods provided in this manual will be described below using the patient's spine as the designated lesion site. Of course, the methods provided in this manual can be used to perform surgery on other areas that need to be treated, and can also avoid damaging the nerve tissue around the bone tissue while ensuring the completion of the bone-shaping surgery. In short, this manual does not specifically limit the bone-shaping site.

[0064] Furthermore, it should be noted that the method provided in this manual is a medical image that can display the patient's lesion site during the surgical preparation stage, thereby assisting medical personnel in observing the lesion site and selecting the surgical area. The personnel involved are typically medical personnel and patients. However, non-medical personnel, i.e., ordinary users, can also use this method to observe the bone tissue to be treated in a patient. For example, in medical education, students can use this method to learn the points that need attention in bone and soft tissue reduction surgery. For ease of explanation, the following descriptions will only focus on medical personnel as users of the method provided in this manual.

[0065] Before bone-shaping surgery, in the preoperative stage, it is necessary to use high-precision image acquisition equipment to capture images of the bone tissue to be treated and the surrounding soft tissue at the lesion site, obtaining the first medical image of the designated area. The reason for acquiring the first medical image in the preoperative stage is that to obtain a first medical image that accurately represents the positional relationship between the bone tissue to be treated and the surrounding soft tissue, high-precision image acquisition equipment is required. However, high-precision image acquisition equipment capable of acquiring such images often cannot capture images of the lesion site while the patient is in the preoperative stage, i.e., the surgical preparation stage.

[0066] Therefore, to ensure the acquisition of a first medical image that accurately characterizes the positional relationship between the bone tissue to be treated and the surrounding soft tissue, it is necessary to acquire images of the patient's lesion site preoperatively using high-precision image acquisition equipment such as magnetic resonance imaging (MRI) or computed tomography (CT). Of course, this manual does not specifically limit the use of high-precision image acquisition equipment; any device capable of acquiring a medical image that accurately characterizes the positional relationship between the bone tissue to be treated and the surrounding soft tissue can be used as a high-precision image acquisition device in this manual. MRI is a commonly used high-precision image acquisition device. Below, we will use an MRI image of the spine (an MRI image of the spine acquired by an MRI machine) as an example to illustrate the positional relationship between the bone tissue to be treated and the surrounding soft tissue.

[0067] Figure 2 This is a schematic diagram of an image provided in this specification that can characterize the positional relationship between the bone tissue to be treated and the soft tissue surrounding the bone tissue.

[0068] Figure 2 The image shown is a transverse MRI image of the spine, containing bone tissue including the vertebral arch, superior articular processes, and inferior articular processes, soft tissue including intervertebral discs, nerve roots, and intraspinal nerve tissue, as well as joint capsules that provide stability and protection. By reconstructing multiple transverse images using specialized image processing software, a three-dimensional image of the patient's lesion can be obtained, which can then be used as the primary medical image.

[0069] The following section will use images to illustrate how the positional relationship between bone tissue and surrounding soft tissue is represented in the image obtained by 3D reconstruction of multiple cross-sectional images.

[0070] Figure 3 This is a schematic diagram of an image representing the positional relationship between the vertebra to be treated and the surrounding soft tissues, as provided in this specification.

[0071] Images of three vertebral bodies and the connecting tissue between two vertebral bodies are shown. The black areas represent cross-sections of the vertebral bodies, which are the basic components of the spine. They are interconnected by intervertebral discs, forming the main structure of the spine. The images also include some soft tissue distributed near the vertebral bodies. Of course, in actual 3D images, soft tissue is not simply distributed near bone tissue; for example, the connection between two vertebral bodies usually also contains some soft tissue. This description is for illustrative purposes only.

[0072] It should be noted that the specific area to be operated on in bone-shaping surgery depends on the patient's condition. For example, if there is an intervertebral disc problem in the patient's spine, bone-shaping surgery may involve the treatment of the vertebral arch. However, the method provided in this manual does not limit the type of bone tissue. The bone tissue to be treated can be determined entirely based on the patient's condition. The method provided in this manual can then be used to determine the location of the bone tissue to be treated relative to the surrounding soft tissue during the surgical preparation stage, thereby determining the actual operable area.

[0073] After acquiring the first medical image of the designated lesion site in the preoperative stage, the second medical image acquired in the surgical preparation stage can be corrected using the first medical image to obtain and display the third medical image of the designated lesion site in the surgical preparation stage. Then, the surgical image area for the bone tissue to be treated can be determined based on the third medical image.

[0074] S102: When the patient is in the surgical preparation stage, an image of the designated lesion site is obtained based on a preset scanning device used in the surgical preparation stage, and used as a second medical image.

[0075] Patients in the pre-operative stage are often in a ready-to-operate position and may be carrying some surgical equipment. The patient's posture and the surgical equipment they are carrying can affect the image acquisition equipment's ability to capture images of the lesion site. Therefore, to allow medical personnel to directly observe the patient's lesion site during the pre-operative stage, it is necessary to use other equipment to acquire images of the lesion site.

[0076] The scanning equipment available during surgical preparation, such as cone-beam computed tomography (CBCT) or spiral computed tomography (SCT), often fails to accurately characterize the positional relationship between the bone tissue to be treated and the surrounding soft tissue. This is because CBCT images of soft tissue have low contrast and limited ability to resolve fine structures and pathological changes within soft tissue. In CBCT images, soft tissue edges may not be clearly defined, and the details and layers of internal structures are less apparent than in MRI images.

[0077] Furthermore, in order for medical personnel to be able to directly observe the patient's lesion site, especially the nerve tissue within the lesion, during the surgical preparation phase, the images acquired during this phase need to accurately display the soft tissue and be able to classify the soft tissue using methods such as semantic segmentation to identify the image region containing the nerve tissue, i.e., the neural region. MRI images can accurately display soft tissue and can also be used for soft tissue classification, but MRI equipment cannot be used in the surgical preparation phase. Even if soft tissue classification is performed on images acquired by equipment available during the surgical preparation phase, it is often difficult to accurately display the image region containing the nerve tissue, making it difficult to identify the image region containing the nerve tissue.

[0078] In order to determine the image region where the nerve tissue in the soft tissue of the patient's lesion site is located during the surgical preparation stage, the image acquired by the first medical image obtained in the preoperative stage, such as the image acquired by the CBCT equipment, can be corrected to obtain an image that can accurately characterize the positional relationship between the bone tissue and soft tissue to be treated. Then, the soft tissue in the image is classified to obtain the image region where the nerve tissue in the soft tissue of the patient's lesion site is located during the surgical preparation stage.

[0079] Below, we will use images to illustrate the second medical image, which is obtained by scanning the patient's spine with a preset CBCT device during the surgical preparation stage.

[0080] Figure 4 This is a schematic diagram of a second medical image obtained from a scanning device, as provided in this specification.

[0081] As can be seen from the image, the second medical image includes the bone tissue to be processed and the surrounding soft tissue. The soft tissue in the second medical image is more disordered, and its boundaries and positions are less precise compared to the first medical image. While the positional relationship between the bone tissue to be processed and the surrounding soft tissue shown in the second medical image is less precise than that in the first medical image, the positional relationship between vertebrae can still be clearly determined for bone tissues such as vertebrae.

[0082] Because the accuracy of the second medical image acquired using CBCT during the surgical preparation phase, which characterizes the positional relationship between the bone and soft tissue to be treated, is lower than that of the first medical image acquired using MRI during the preoperative phase, it is necessary to correct the second medical image acquired during the surgical preparation phase using the first medical image. This correction aims to determine the positional relationship between the bone and soft tissue to be treated during the surgical preparation phase, thereby obtaining and displaying a third medical image that accurately characterizes the positional relationship between the bone and soft tissue at the designated lesion site during the surgical preparation phase.

[0083] S103: Using the first medical image, the positional relationship between the bone tissue to be processed and the soft tissue in the second medical image is corrected to obtain and display a third medical image of the designated lesion site of the patient in the surgical preparation stage.

[0084] It should be noted that the patient's posture before surgery is often different from that during surgical preparation. If the second medical image is to be corrected using the first medical image, methods such as gray-scale difference and Euclidean distance, mutual information methods, or feature extraction and registration methods based on feature points, such as SIFT (Scale-Invariant Feature Transform), can be used. Of course, deep learning registration and other methods can also be used. This manual does not impose any specific restrictions.

[0085] Taking the SIFT feature extraction and registration method as an example, the first and second medical images need to be preprocessed before registration, such as grayscale conversion, noise reduction, and smoothing, in order to improve the stability and accuracy of the images.

[0086] Subsequently, feature extraction was performed on the preprocessed first and second medical images to determine the locations of key points in each image. Each key point location exhibits local maxima or minima at different scales. Then, interpolation was used to precisely determine the location of each key point, and its scale was calculated. Next, the corresponding principal orientation was calculated for each key point, and a fixed-size neighborhood was constructed around each key point. By statistically calculating the gradient information within this neighborhood, a 128-dimensional SIFT feature vector was generated. This vector is robust to illumination, scale changes, and rotations within a certain range. The scale includes the physical scale, image resolution, feature scale, and image pyramid scale.

[0087] Furthermore, using a feature matching algorithm, each SIFT feature vector in the first medical image is matched with the SIFT feature vectors contained in the second medical image to obtain initial matching pairs. These initial matching pairs are then filtered to obtain reliable matching pairs. Based on these reliable matching pairs, geometric transformation parameters between the first and second medical images are calculated using methods such as direct linear transformation. The calculated set transformation parameters are then used to determine the positional relationship between the bone and soft tissues to be processed. This allows the second medical image to be corrected so that it accurately represents the positional relationship between the bone and soft tissues. The corrected second medical image is then used as the third medical image.

[0088] The third medical image is an image of the designated lesion site when the patient is in the surgical preparation stage. Displaying this image allows medical personnel to observe the designated lesion site of the patient. The third medical image includes the coordinates of the bone tissue, the coordinates of the soft tissue surrounding the bone tissue, and the coordinates of the optical ball end.

[0089] S104: In response to a selection operation performed by the user on the third medical image, an initial surgical image region for the bone tissue to be processed is determined in the third medical image.

[0090] S105: Adjust the initial surgical image region according to the image region where the nerve tissue in the soft tissue is located in the third medical image, so as to determine and display the actual operable image region in the third medical image.

[0091] During the surgical preparation phase, after observing the displayed third-party medical image, medical personnel can gain a direct understanding of the patient's lesion site. They can then select areas within the third-party image to identify the initial surgical image region for the bone tissue to be treated. It should be noted that the third-party medical image is a three-dimensional image. To facilitate image region selection, the three-dimensional image and its corresponding cross-sections can be displayed separately. After selecting the image region for each cross-section in the three-dimensional image, the corresponding selected image region will automatically appear in the three-dimensional image. This selected image region in the three-dimensional image is the initial surgical image region for the bone tissue to be treated.

[0092] After determining the initial surgical image region, the initial surgical image region can be detected to ensure that the initial surgical image region does not contain the image region where the nerve tissue is located.

[0093] Specifically, it is necessary to detect the image region containing the soft tissue in the third medical image to determine the image region containing the neural tissue within the soft tissue. Detection methods can include thresholding, watershed analysis, region growing, or semantic analysis based on neural networks, etc. The goal is simply to determine the image region containing the neural tissue in the soft tissue of the third medical image. This specification does not impose specific limitations; the goal is simply to determine the image region containing the neural tissue in the soft tissue of the third medical image. It should be noted that it is entirely possible to first determine the image region containing the neural tissue in the first medical image, and then, after correcting the second medical image using the first medical image, directly obtain the image region containing the neural tissue in the soft tissue of the third medical image.

[0094] After identifying the image region containing the nerve tissue in the soft tissue within the third medical image, the image region containing the nerve tissue can be identified from the initial surgical image region and designated as the nerve image region. This nerve image region is then removed from the initial surgical image region, and the remaining image region is determined as the actual operable image region. Performing bone-shaping surgery based on this actual operable region can minimize the risk of accidental damage to the nerve tissue during the procedure.

[0095] Considering the potential for unforeseen factors during bone-grinding surgery, the surgical tool's effective range might exceed the actual operable image area, potentially damaging nerve tissue. Therefore, a third medical image can be used to identify the region containing nerve tissue outside the initial surgical image area. This region is designated as the nerve image area outside the initial surgical image area. Then, using the center point of this nerve image area as the center and a preset safety distance threshold as the radius, a surgical avoidance image area is defined. The intersection of the surgical avoidance image area and the initial surgical image area is determined, and this intersection area is removed from the initial surgical image area. The remaining image area in the initial surgical image area is then defined as the actual operable image area. This method maximizes the protection of nerve tissue during actual surgery. The preset safety distance threshold can be determined based on the size of the surgical tool's head and the size of the nerve tissue, or it can be a fixed value.

[0096] In practical applications, in addition to using the center point of the neural image region as the center, several points can be selected on the edge of the neural image region. Then, for each point, each surgical avoidance image region is determined. The union of all surgical avoidance image regions is taken as the total surgical avoidance image region. Then, the intersection region of the total surgical avoidance image region and the initial surgical image region is determined. Then, this intersection region is removed from the initial surgical image region, and the remaining image region in the initial surgical image region is determined as the actual operable image region.

[0097] In summary, this specification provides two methods for determining the actual operable image region. These two methods do not conflict with each other. In practical applications, it is entirely possible to simultaneously detect the image region containing neural tissue located outside the initial surgical image region and the image region containing neural tissue located within the initial surgical image region, and then determine the actual operable image region based on the detected image regions.

[0098] Once the actual operable image area is determined, it can be displayed, and bone-shaping surgery can be performed based on this actual operable image area.

[0099] For surgeries requiring the acquisition of the actual location of the patient's bone tissue and surgical instruments, this specification provides a method for locating bone tissue and surgical instruments based on an optical tracer and a preset optical tracker. The optical tracer is used to optically track the optical tracker using light reflected from the surface of its optical ball end, thereby determining the three-dimensional coordinates of the optical tracker. The optical tracker includes a fixed end and an optical ball end, which are rigidly connected to maintain the relative positional relationship between the optical ball end and the object to be located. The optical tracer optically tracks the optical tracker using light reflected from the surface of its optical ball end, and the resulting three-dimensional coordinates of the optical ball end can be used as the three-dimensional coordinates of the optical tracker.

[0100] Specifically, taking the spine as the lesion site as an example, when the patient is in the surgical preparation stage, an optical tracker can be pre-installed on the patient's spine as the first tracker. The fixed end of the first tracker is fixed to the patient's bone tissue, and the optical ball end is placed outside the patient's body, usually on the patient's skin. It should be noted that multiple optical trackers can be pre-installed on the patient's lesion site, but because installing optical trackers on the lesion site will also put pressure on the lesion site, usually only one optical tracker is used. However, for large-scale surgeries, multiple optical trackers can be installed on the patient's lesion site. Since surgeries targeting bone tissue are often minimally invasive, and for minimally invasive bone reshaping surgery, often only one optical tracker is needed. Therefore, this instruction manual only uses the example of installing only one optical tracker on the patient's lesion site. In addition, for the optical tracer used to optically track the optical tracker, other optical tracking devices such as dedicated optical tracking systems or other optical tracking devices can also be used, and this instruction manual does not make specific restrictions.

[0101] The patient's spine, equipped with an optical tracker, is scanned using a pre-set scanning device during surgical preparation. The resulting second medical image includes the optical ball of the first tracker. Based on this second medical image, the positional relationship between the optical ball of the first tracker and the patient's bone tissue to be treated can be determined. Then, the first tracker is optically tracked in real time using an optical tracer to obtain its real-time three-dimensional coordinates. Furthermore, if the three-dimensional coordinates of the optical ball of the first tracker change relative to the time the second medical image is obtained, it is determined that the patient's posture has changed. Then, based on the changed position of the optical ball of the first tracker and the determined positional relationship between the optical ball of the first tracker and the patient's bone tissue to be treated, the coordinates of the patient's bone tissue to be treated after the change in patient posture can be obtained.

[0102] Furthermore, based on the determined relative positional relationship between the bone tissue and soft tissue to be treated, and the coordinates of the bone tissue to be treated after changes in the patient's posture, the image regions containing the bone tissue and soft tissue in the third medical image can be corrected. The relative positional relationship between the bone tissue and soft tissue can be obtained during the correction of the second medical image; for example, the positional relationship between the bone tissue and soft tissue can be determined using the SIFT feature extraction and registration method.

[0103] An optical tracker is installed on the surgical instrument as a second tracker. This second tracker can also determine the actual coordinates of the surgical instrument, providing its actual position. Specifically, the second tracker is optically tracked by an optical tracer to obtain the three-dimensional coordinates of its optical sphere. Based on these three-dimensional coordinates and the pre-determined positional relationship between the optical sphere and the surgical instrument, the actual coordinates of the surgical instrument are determined. Furthermore, determining the actual coordinates of the bone tissue to be treated using the first tracker and the actual coordinates of the surgical instrument using the second tracker can be performed simultaneously. If the first tracker is optically tracked in real-time by the optical tracer, the real-time coordinates of the patient's bone tissue to be treated can be obtained; similarly, if the second tracker is optically tracked in real-time by the optical tracer, the real-time coordinates of the surgical instrument can be obtained.

[0104] Since the first and second trackers are often the same optical tracer for optical tracking, the coordinates of the patient's bone tissue to be treated and the coordinates of the surgical tool, determined by the first and second trackers, are in the same coordinate system. This allows for real-time determination of the relative positional relationship between the surgical tool and the bone tissue. Furthermore, based on factors such as the radius of the surgical tool's head (e.g., the radius of the grinding head) and the determined relative positional relationship between the surgical tool and the bone tissue, the area of ​​action of the surgical tool at the lesion site can be determined and displayed in the actual operable image area. Of course, even if the optical tracer used for optical tracking of the first and second optical trackers are not the same, the coordinates of both optical tracers can be determined first, and then, based on these coordinates, the coordinates of the patient's bone tissue to be treated and the coordinates of the surgical tool can be transformed into the same coordinate system.

[0105] The actual coordinates of surgical tools can also be determined by other methods. For example, for surgical tools mounted on a surgical robot, the actual coordinates of the surgical robot can be obtained first, and then the actual coordinates of the surgical tool can be determined based on the positional relationship between the surgical robot and the mounted surgical tool, as well as the parameter data corresponding to the fixed model of the surgical tool.

[0106] Once the actual coordinates, or actual position, of the surgical tool are determined, they can be unified with the coordinates of the patient's bone tissue to be treated into the same coordinate system. Then, based on the positional relationship between the surgical tool (i.e., the drill) and its grinding head, the real-time area of ​​action of the surgical tool at the designated lesion site can be determined, which is the real-time bone-grinding range of the grinding head. Besides the method provided in this manual for determining the area of ​​action of the surgical tool at the designated lesion site within the actual operable image area, there are several other methods that can be used, which will not be elaborated upon here.

[0107] In summary, the purpose of determining the actual position of the surgical tool and the actual position of the patient's bone tissue to be treated is to determine the relative positional relationship between the surgical tool and the patient's lesion area, and then to perform surgery based on this relative positional relationship or using an image to display the relative positional relationship. Furthermore, as long as the coordinates of the surgical tool and the coordinates of the corresponding portion of the actual operable image area within the lesion area can be determined in the same coordinate system, surgery can be performed using the surgical tool, and the area of ​​action of the surgical tool in the designated lesion site can be determined. The method for determining the relative positional relationship based on an optical tracer provided in this manual is only one method for determining relative positional relationships, and this manual does not impose specific limitations on the method used to determine relative positional relationships. It should be noted that the method for determining positional relationships provided in this manual is not limited to determining the positional relationship between the bone tissue to be treated and the surgical tool; it can also determine, for example, the positional relationship between other soft tissues to be protected in the patient's lesion area and the surgical equipment, and this manual does not impose specific limitations on this. In addition, the image area that has been identified as the working area of ​​the surgical tool in the actual operable image area can be used as the image area of ​​the completed bone reshaping. The image areas of the completed bone reshaping and the image areas of the unrefined bone reshaping can be marked in the actual operable image area by using two different colors in a preset display mode.

[0108] Using the above method, medical personnel, i.e., users, can correct the second medical image obtained in the preoperative preparation stage using the first medical image acquired in the preoperative stage to obtain a third medical image of the lesion site of the patient in the surgical preparation stage. The accuracy of the positional relationship between the bone tissue and soft tissue to be treated represented by this third medical image is higher than that of the second medical image. Then, the image area selected by the user can be received from the third medical image as the initial surgical image area. The initial surgical image area is then adjusted by the image area where the nerve tissue in the soft tissue is located in the determined third medical image to obtain the actual operable image area for the bone grinding surgery process, which can avoid damage to the nerve tissue, and the actual operable image area is displayed.

[0109] Below, based on the same approach as the image display method, we will use images to illustrate a surgical system for performing surgery.

[0110] Figure 5 This is a schematic diagram of a surgical system provided in this specification.

[0111] The patient is equipped with scanning devices, display devices, a surgical robot, optical trackers, and optical tracers. The patient is currently in the surgical preparation phase.

[0112] The scanning device is used to scan the designated lesion site of the patient during the surgical preparation stage, obtain an image of the designated lesion site as a second medical image, and send the second medical image to the display device.

[0113] After receiving the second medical image sent by the scanning device, the display device acquires a high-precision image of the bone tissue to be treated and the surrounding soft tissue at the designated lesion site, obtained through a high-precision image acquisition device during the preoperative stage, as the first medical image. Then, using the first medical image, the positional relationship between the bone tissue to be treated and the soft tissue in the second medical image is corrected to obtain a third medical image of the lesion site during the patient's surgical preparation stage, which is displayed on the screen. Subsequently, in response to the user selecting a portion of the third medical image, the user-selected area is used as the initial surgical image area for the bone tissue to be treated. Then, based on the image area of ​​the nerve tissue in the soft tissue in the third medical image, the initial surgical image area is adjusted to determine the actual operable image area in the third medical image and display it on the screen. Surgical instructions are generated based on the actual operable image area and sent to the surgical robot. The accuracy of the positional relationship between the bone tissue to be treated and the soft tissue represented by the first medical image is higher than that of the second medical image.

[0114] Specifically, in determining the actual operable image area, the display device can identify the image area where the nerve tissue is located from the initial surgical image area, use it as the nerve image area, remove the nerve image area from the initial surgical image area, and determine the remaining image area in the initial surgical image area as the actual operable image area.

[0115] The display device can also identify the image region containing nerve tissue outside the initial surgical image region from the third medical image, and then divide the surgical avoidance image region with the center point of the nerve image region as the center and the preset safe distance threshold as the radius. Then, the intersection area of ​​the surgical avoidance image region and the initial surgical image region is determined, the intersection area is removed from the initial surgical image region, and the remaining image region in the initial surgical image region is determined as the actual operable image region.

[0116] After receiving surgical instructions from the display device, the surgical robot controls the surgical tools according to the instructions to perform surgery on the patient's designated lesion site on the operating table. Of course, the patient can be located on other facilities, such as an operating chair, a standing position, or a mobile operating table; this instruction manual does not impose specific limitations.

[0117] The optical tracker includes a fixed end and an optical ball end. The optical tracer performs optical tracking based on the light reflected from the optical ball end of the optical tracker, thereby obtaining the three-dimensional coordinates of the optical ball end as the three-dimensional coordinates of the optical tracker. Based on these three-dimensional coordinates, the actual coordinates of the object to be located are determined. During surgery, the display device can use the optical tracer to track the optical tracker mounted on the surgical tool, determining the actual coordinates of the surgical tool as its actual position. This allows the surgical tool's area of ​​action within the designated lesion site to be marked in the actual operable image area. The surgical tool is mounted on a surgical robot. Furthermore, the area of ​​action of the surgical tool can be determined based on its real-time position determined by optical tracking. The image area within the actual operable image area that has been identified as the area of ​​action of the surgical tool can also be used as the image area of ​​completed bone reshaping. Two different colors are used in a preset display method to mark the image areas of completed bone reshaping and those of unrefined bone reshaping within the actual operable image area.

[0118] Furthermore, an optical tracker can be placed at the patient's lesion site (spine). During the surgery, the optical tracker placed at the patient's lesion site can be optically tracked in real time by an optical tracer. When the coordinates at the optical ball end change, it can be determined that the patient's posture has changed. Then, the coordinates at the optical ball end are sent to the display device. The display device can determine the coordinates of the bone and soft tissues after the patient's posture has changed based on the coordinates at the optical ball end. Based on the coordinates of the bone and soft tissues after the patient's posture has changed, the display device can correct and display the actual operable image area.

[0119] The display device can also generate surgical instructions during the surgical preparation stage by using the coordinates of the patient's bone tissue to be treated and the coordinates of the surgical tools obtained by the optical tracer, so as to control the surgical robot to perform surgery on the patient's actual operable area.

[0120] It should be noted that the methods for determining the actual positions of the first medical image, the second medical image, the third medical image, the surgical instruments, and the bone tissue to be processed have been explained and exemplified in the image display method provided in this specification, and will not be repeated here.

[0121] Figure 6 This is a schematic diagram illustrating the progress of bone reshaping in an actual surgically operable image area, as provided in this specification.

[0122] The display device and its integrated display screen show the actual surgically operable image area. During the surgery, the display device acquires the actual position of the surgical instruments and marks the area of ​​action of the surgical instruments at the designated lesion site within the actual surgically operable image area. This area is marked with diagonal lines. The image area within the actual surgically operable image area that has been identified as the area of ​​action of the surgical instruments is considered the completed bone-reduction image area, i.e., the black image area. Conversely, the white areas within the actual surgically operable image area are considered the incomplete bone-reduction image areas. This display method allows for a direct visual representation of the progress of the bone-reduction surgery. Furthermore, the percentage of bone reduction completed can be determined and displayed by comparing the size ratio between the completed bone-reduction image area and the actual surgically operable image area.

[0123] Through the aforementioned system, bone-grinding surgery targeting the patient's lesion site can be performed more effectively. Medical personnel can intuitively understand the patient's lesion site through the third medical image displayed on the display device, and select an initial surgical image area based on this. Then, the display device can optimize the initial surgical image area to obtain the actual operable area, so as to avoid damage to nerve tissue during bone-grinding surgery performed based on the initial surgical image area. Furthermore, the display device can generate surgical instructions based on the actual operable area and send the surgical instructions to the surgical robot, so that the surgical robot can perform the surgery according to the surgical instructions. In addition, the display device can also obtain the actual position of the surgical tools during the operation, mark the area of ​​action of the surgical tools in the designated lesion site in the actual operable image area, and display the bone-grinding progress in real time. While ensuring the success of the bone-shaping surgery, damage to the nerve tissue surrounding the bone tissue is avoided. This allows medical staff to more intuitively understand the condition of the patient's lesion area. The surgery is automated through a surgical robot. Furthermore, even if the patient's posture changes during the surgery, the coordinates of the bone and soft tissue to be treated can still be determined and displayed through optical trackers and optical tracers after the change in the patient's posture, ensuring the normal progress of the surgery.

[0124] The above describes one or more image display methods and surgical systems based on the same concept. This manual also provides corresponding devices and storage media.

[0125] Figure 7 This is a schematic diagram of the structure of an image display device provided in an embodiment of this specification. The device includes:

[0126] The acquisition module 701 is used to acquire a first medical image of a specified lesion site of a patient, wherein the first medical image is obtained by acquiring the bone tissue to be treated and the soft tissue around the bone tissue to be treated at the specified lesion site through a high-precision image acquisition device in the preoperative stage.

[0127] The scanning module 702 is used to obtain an image of the designated lesion site based on a preset scanning device used in the surgical preparation stage when the patient is in the surgical preparation stage, as a second medical image. The accuracy of the positional relationship between the bone tissue to be processed and the soft tissue represented by the second medical image is lower than that of the first medical image.

[0128] Display module 703 is used to correct the positional relationship between the bone tissue to be processed and the soft tissue in the second medical image through the first medical image, and to obtain and display a third medical image of the designated lesion site when the patient is in the surgical preparation stage;

[0129] The first determining module 704 is configured to determine an initial surgical image region for the bone tissue to be processed in the third medical image in response to a selection operation performed by a user on the third medical image.

[0130] The second determining module 705 is used to adjust the initial surgical image region based on the image region where the nerve tissue in the soft tissue is located in the third medical image, so as to determine and display the actual operable image region in the third medical image.

[0131] Optionally, the second determining module 705 is specifically used for,

[0132] The image region containing the nerve tissue is identified from the initial surgical image region and designated as the nerve image region; the nerve image region is removed from the initial surgical image region, and the remaining image region in the initial surgical image region is determined as the actual operable image region.

[0133] Optionally, the second determining module 705 is specifically used for,

[0134] The image region containing nerve tissue located outside the initial surgical image region is identified from the third medical image and designated as the nerve image region; a surgical avoidance image region is determined based on the nerve image region; the initial surgical image region is adjusted based on the surgical avoidance image region to determine the actual operable image region.

[0135] Optionally, the second determining module 705 is specifically used for,

[0136] The surgical avoidance image region is divided into areas with the center point of the neural image region as the center and a preset safe distance threshold as the radius.

[0137] Optionally, the second determining module 705 is specifically used for,

[0138] Determine the intersection region between the surgical avoidance image region and the initial surgical image region; remove the intersection region from the initial surgical image region, and determine the remaining image region in the initial surgical image region as the actual operable image region.

[0139] Optionally, the device further includes:

[0140] The marking module 706 is used to mark the area of ​​action of the surgical tool in the designated lesion site in the actual operable image area according to the actual position of the surgical tool.

[0141] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 Provided image display methods.

[0142] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0143] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0144] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0145] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0146] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0147] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0148] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0149] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0150] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0151] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0152] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0153] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0154] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0155] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0157] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0158] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. An image display method, characterized in that, include: Acquire a first medical image of a designated lesion site of a patient, wherein the first medical image is obtained by acquiring the bone tissue to be treated and the soft tissue surrounding the bone tissue at the designated lesion site using a high-precision image acquisition device during the preoperative stage. While the patient is in the surgical preparation stage, an image of the designated lesion site is obtained based on a pre-set scanning device used in the surgical preparation stage, which serves as a second medical image. The accuracy of the positional relationship between the bone tissue to be treated and the soft tissue represented by the second medical image is lower than that of the first medical image. Using the first medical image, the positional relationship between the bone tissue to be processed and the soft tissue in the second medical image is corrected to obtain and display a third medical image of the designated lesion site when the patient is in the surgical preparation stage; In response to a user's selection operation on the third medical image, an initial surgical image region for the bone tissue to be processed is determined in the third medical image; The initial surgical image region is adjusted based on the image region where the nerve tissue in the soft tissue is located in the third medical image, so as to determine and display the actual operable image region in the third medical image.

2. The method as described in claim 1, characterized in that, Based on the image region where the nerve tissue in the soft tissue is located in the third medical image, the initial surgical image region is adjusted to determine the actual operable image region in the third medical image, specifically including: The image region containing the neural tissue is identified from the initial surgical image region and designated as the neural image region; The neural image region is removed from the initial surgical image region, and the remaining image region in the initial surgical image region is determined as the actual operable image region.

3. The method as described in claim 1, characterized in that, Based on the image region where the nerve tissue in the soft tissue is located in the third medical image, the initial surgical image region is adjusted to determine the actual operable image region in the third medical image, specifically including: The image region containing the neural tissue located outside the initial surgical image region is identified from the third medical image and designated as the neural image region; Based on the neural image region, the surgical avoidance image region is determined; Based on the surgical avoidance image region, the initial surgical image region is adjusted to determine the actual operable image region.

4. The method as described in claim 3, characterized in that, Based on the neural image region, the surgical avoidance image region is determined, specifically including: The surgical avoidance image region is divided into areas with the center point of the neural image region as the center and a preset safe distance threshold as the radius.

5. The method as described in claim 3, characterized in that, Based on the surgical avoidance image region, the initial surgical image region is adjusted to determine the actual operable image region, specifically including: Determine the intersection region of the surgical avoidance image region and the initial surgical image region; The intersection region is removed from the initial surgical image region, and the remaining image region in the initial surgical image region is determined as the actual operable image region.

6. The method as described in claim 1, characterized in that, The method further includes: Based on the actual position of the surgical tool, mark the area of ​​action of the surgical tool in the designated lesion site within the actual operable image area.

7. A surgical system, characterized in that, The system includes: a scanning device, a display device, and a surgical robot; The scanning device is used to scan a designated lesion site of the patient during the surgical preparation stage, obtain an image of the designated lesion site as a second medical image, and send the second medical image to the display device; The display device is configured to receive the second medical image sent by the scanning device, and acquire a high-precision image obtained by a high-precision image acquisition device during the preoperative stage of acquiring the bone tissue to be treated and the soft tissue surrounding the bone tissue at the designated lesion site, as the first medical image. Using the first medical image, the positional relationship between the bone tissue to be treated and the soft tissue in the second medical image is corrected to obtain and display a third medical image of the designated lesion site when the patient is in the surgical preparation stage. In response to a user's selection operation on the third medical image, an initial surgical image region for the bone tissue to be treated is determined in the third medical image. Based on the image region where the nerve tissue in the soft tissue is located in the third medical image, the initial surgical image region is adjusted to determine and display an actual operable image region in the third medical image. Surgical instructions are generated based on the actual operable image region and sent to the surgical robot. The accuracy of the positional relationship between the bone tissue to be treated and the soft tissue represented by the first medical image is higher than that of the second medical image. The surgical robot is used to receive the surgical instructions sent by the display device and control the surgical tools to perform surgery on the designated lesion site according to the surgical instructions.

8. The system as described in claim 7, characterized in that, The display device is configured to identify the image region containing the nerve tissue from the initial surgical image region as the nerve image region, remove the nerve image region from the initial surgical image region, and determine the remaining image region in the initial surgical image region as the actual operable image region.

9. The system as described in claim 7, characterized in that, The display device is used to identify, from the third medical image, an image region containing nerve tissue located outside the initial surgical image region, as the nerve image region; to determine a surgical avoidance image region based on the nerve image region; and to adjust the initial surgical image region based on the surgical avoidance image region to determine the actual operable image region.

10. The system as described in claim 9, characterized in that, The display device is used to divide the surgical avoidance image region with the center point of the neural image region as the center and a preset safe distance threshold as the radius.

11. The system as described in claim 9, characterized in that, The display device is used to determine the intersection area of ​​the surgical avoidance image area and the initial surgical image area, remove the intersection area from the initial surgical image area, and determine the remaining image area in the initial surgical image area as the actual operable image area.

12. The system as described in claim 7, characterized in that, The system also includes: an optical tracker and an optical tracer; The display device is used to mark the area of ​​action of the surgical tool in the specified lesion site in the actual operable image area, based on the actual position of the surgical tool determined by the optical tracker installed on the surgical robot using the optical tracer.

13. The system as described in claim 7, characterized in that, The display device is used during surgery to mark the image areas where bone reshaping has been completed and the image areas where bone reshaping has not yet been performed in the actual operable image area using different colors in a specified display manner.

14. An image display device, characterized in that, include: The acquisition module is used to acquire a first medical image of a specified lesion site of a patient, wherein the first medical image is obtained by acquiring the bone tissue to be treated and the soft tissue around the bone tissue to be treated at the specified lesion site through a high-precision image acquisition device in the preoperative stage. The scanning module is used to obtain an image of the designated lesion site based on a preset scanning device used in the surgical preparation stage when the patient is in the surgical preparation stage, as a second medical image. The accuracy of the positional relationship between the bone tissue to be processed and the soft tissue represented by the second medical image is lower than that of the first medical image. The display module is used to correct the positional relationship between the bone tissue to be processed and the soft tissue in the second medical image based on the first medical image, and to obtain and display a third medical image of the designated lesion site when the patient is in the surgical preparation stage. The first determining module is configured to determine an initial surgical image region for the bone tissue to be processed in the third medical image in response to a selection operation performed by the user on the third medical image. The second determining module is used to adjust the initial surgical image region based on the image region where the nerve tissue in the soft tissue is located in the third medical image, so as to determine and display the actual operable image region in the third medical image.

15. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 6.

Images