Image processing method for medical assistance, c-arm machine, electronic device and medium

Abstract

The application discloses an image processing method for medical assistance, a C-arm machine, electronic equipment and a medium, and relates to the medical field. The method is applied to the C-arm machine, the C-arm machine comprises an X-ray shooting module and a visible light camera, and comprises the following steps: acquiring an X-ray image based on the X-ray shooting module, and acquiring a real-time visible light image based on the visible light camera, wherein the shooting angles of the X-ray image and the real-time visible light image are consistent, and the X-ray image and the real-time visible light image both comprise the image of a calibration tool; matching and fusing the X-ray image and the real-time visible light image based on the position information of the calibration tool to obtain a real-time fusion image, and displaying the real-time fusion image; and guiding a surgical operation based on the displayed real-time fusion image and a target region. The method of the application generates a fusion image by combining an X-ray image and a visible light image in surgery, so that medical personnel can intuitively observe the physiological structure of a patient and the situation of surgical equipment through the fusion image, and medical assistance is provided.

Description

Technical Field

[0001] This application relates to the fields of medical technology and medical assistance, and in particular to an image processing method, C-arm machine, electronic device and medium for medical assistance. Background Technology

[0002] In surgeries using C-arm imaging for positioning, such as percutaneous spinal surgery, ensuring the needle accurately punctures the target point along the designated path is crucial for success. The standard procedure involves attaching a positioning net to the patient, taking anteroposterior and lateral views, and then marking the puncture point and direction on the skin based on the target point's position on the net. The puncture is then performed according to these markings. However, because the skin markings only indicate the top-down view and not the side-down view, there is no guidance during the puncture. Furthermore, after each puncture, repeat anteroposterior and lateral X-rays are required to confirm correct placement. Traditional puncture procedures are not highly precise and are not safe. Summary of the Invention

[0003] Therefore, the purpose of this application is to provide an image processing method, C-arm machine, electronic device, storage medium and computer program product for medical assistance, which combines X-ray images and visible light images to generate a fused image during surgery, so that medical personnel can intuitively observe the patient's physiological structure and the condition of the surgical equipment through the fused image, thus providing medical assistance.

[0004] One embodiment of this application provides an image processing method for medical assistance, applied to a C-arm machine. The C-arm machine includes an X-ray imaging module and a visible light camera. The method includes: acquiring an X-ray image based on the X-ray imaging module and acquiring a real-time visible light image based on the visible light camera, wherein the X-ray image and the real-time visible light image have the same shooting angle, and both the X-ray image and the real-time visible light image include an image of a calibration tool; matching and fusing the X-ray image and the real-time visible light image based on the position information of the calibration tool to obtain a real-time fused image, and displaying the real-time fused image, which includes a target region; and guiding surgical operations based on the displayed real-time fused image and the target region.

[0005] For example, the X-ray image includes a first-angle X-ray image and a second-angle X-ray image, and the real-time visible light image includes a first-angle real-time visible light image and a second-angle real-time visible light image, wherein at least one of the first angle and the second angle is adjustable; the step of matching and fusing the X-ray image and the real-time visible light image based on the position information of the calibration tool to obtain a real-time fused image, and displaying the real-time fused image, includes: matching and fusing the first-angle X-ray image and the first-angle real-time visible light image based on the position information of the calibration tool to obtain a first-angle real-time fused image, and displaying the first-angle real-time fused image; and / or, matching and fusing the second-angle X-ray image and the second-angle real-time visible light image based on the position information of the calibration tool to obtain a second-angle real-time fused image, and displaying the second-angle real-time fused image.

[0006] For example, the real-time fused image includes an image of a puncture needle, the surgical operation includes a puncture operation, and the surgical operation guided based on the displayed real-time fused image and the target area includes: identifying the puncture needle in the real-time fused image to obtain a puncture needle identification result; and guiding the puncture operation based on the displayed real-time fused image, the target area, and the puncture needle identification result.

[0007] For example, the identification result of the puncture needle includes the position information of the target segment of the puncture needle located outside the body and the position information of the extension line of the puncture needle; the identification of the puncture needle in the real-time fused image to obtain the identification result of the puncture needle includes: identifying the puncture needle in the real-time fused image to obtain the position information of the target segment of the puncture needle located outside the body; and obtaining the position information of the extension line of the puncture needle based on the position information of the target segment.

[0008] For example, the identification result of the puncture needle includes the position information of the target segment of the puncture needle outside the body, the position information of the extension line of the puncture needle, and the position information of the needle tip. The step of identifying the puncture needle in the real-time fused image to obtain the identification result of the puncture needle includes: identifying the puncture needle in the real-time fused image to obtain the position information of the target segment of the puncture needle outside the body; determining the insertion depth of the puncture needle based on the position information of the target segment; and determining the position information of the needle tip based on the position information of the target segment and the insertion depth of the puncture needle.

[0009] For example, the C-arm machine also includes a display screen, and the method further includes: displaying the identification result of the puncture needle based on the display screen, and displaying the real-time fused image.

[0010] For example, guiding the puncture operation based on the displayed real-time fused image, the target area, and the identification result of the puncture needle includes prompting the completion of the puncture operation when the needle tip position information indicates that the needle tip position has reached the target area.

[0011] For example, the method further includes: issuing a prompt message when the distance between the needle tip and the target area is less than a preset distance, wherein the prompt message indicates that the puncture needle is about to reach the target area.

[0012] Another embodiment of this application provides a C-arm machine, comprising: an X-ray imaging module including a flat panel detector and a radiation source disposed opposite to the flat panel detector, the X-ray imaging module being used to capture X-ray images; a C-arm, one end of the C-arm being connected to the flat panel detector and the other end of the C-arm being connected to the radiation source; the C-arm being slidably disposed on a side column of the C-arm; at least two visible light cameras, the at least two visible light cameras being disposed on the C-arm or the side column of the C-arm, adapted to capture real-time visible light images from a preset angle corresponding to the X-ray images, at least one of the at least two visible light cameras being position-adjustable; a processing device, the processing device being connected to the X-ray imaging module and the visible light cameras, the processing device being used to match and fuse the X-ray images and the real-time visible light images to generate a real-time fused image, the real-time fused image including a target area, and guiding surgical operations based on the displayed real-time fused image and the target area; and a display screen, the display screen being connected to the processing device, the display screen being used to display the real-time fused image.

[0013] For example, the number of visible light cameras is 2. The first visible light camera is installed at one end of the C-arm and is adapted to capture real-time visible light images from an adjusted angle, including an orthogonal position. The second visible light camera is fixed on the side column of the C-arm and is used to capture real-time visible light images from the side of the operating table.

[0014] For example, the steps used to implement the method described above;

[0015] The first visible light camera is mounted on one end of the C-arm and rotates with the C-arm, and is located on one side of the flat panel detector; the second visible light camera is fixed on the frame of the C-arm and is used to photograph the patient from one side of the operating table.

[0016] The C-arm rotates, causing the X-ray imaging module to scan around the patient to obtain three-dimensional CT data, which includes the target.

[0017] The processing device is used for:

[0018] Based on the multiple positions rotated to by the C-arm scanning, the CT data is projected along the radial direction of the scanning trajectory to obtain X-ray images from multiple angles;

[0019] Based on multiple X-ray images, the target is identified and its corresponding target region is obtained. The X-ray image corresponding to the largest target region is taken as the first angle X-ray image.

[0020] Based on the shooting angle corresponding to the second visible light camera, the CT data is projected to obtain an X-ray image at the second angle;

[0021] The C-arm rotates to adjust the first visible light camera to the shooting angle corresponding to the largest target area X-ray image.

[0022] A first visible light camera captures a real-time visible light image from the first angle, and a second visible light camera captures a real-time visible light image from the second angle.

[0023] Another embodiment of this application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method of any of the above embodiments.

[0024] Another embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the above embodiments.

[0025] Another embodiment of this application provides a computer program product, which includes instructions that, when executed by a processor of a computer device, enable the computer device to perform the steps of the method described in any of the above embodiments.

[0026] In the above embodiments, the image processing method for medical assistance is applied to a C-arm machine, which includes an X-ray imaging module and a visible light camera. The method includes: acquiring an X-ray image based on the X-ray imaging module and acquiring a real-time visible light image based on the visible light camera, wherein the X-ray image and the real-time visible light image have the same shooting angle, and both the X-ray image and the real-time visible light image include an image of a calibration tool; matching and fusing the X-ray image and the real-time visible light image based on the position information of the calibration tool to obtain a real-time fused image, and displaying the real-time fused image; and guiding surgical operations based on the displayed real-time fused image and a target area. The method of this invention combines X-ray images and visible light images during surgery to generate a fused image, allowing medical personnel to intuitively observe the patient's physiological structure and the status of surgical equipment through the fused image, thus providing medical assistance. Attached Figure Description

[0027] Figure 1A schematic diagram of a conventional puncture procedure provided for an embodiment of this application;

[0028] Figure 2 A flowchart illustrating an image processing method for medical assistance provided in an embodiment of this application;

[0029] Figure 3 A schematic diagram of a C-arm machine provided for an embodiment of this application;

[0030] Figure 4 A schematic diagram of an orthogonal visible light image and an orthogonal X-ray image provided for embodiments of this application;

[0031] Figure 5 Schematic diagrams of lateral visible light images and lateral X-ray images provided for embodiments of this application;

[0032] Figure 6 A schematic diagram of an orthogonal real-time fused image and a lateral real-time fused image provided for embodiments of this application;

[0033] Figure 7 A flowchart for determining whether a surgical operation has been completed based on real-time fused images, provided as an embodiment of this application;

[0034] Figure 8 A flowchart for recognizing real-time fused images provided as an embodiment of this application;

[0035] Figure 9 A flowchart for recognizing real-time fused images provided as another embodiment of this application;

[0036] Figure 10 A block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0037] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0038] In surgeries that utilize C-arm positioning, such as percutaneous spinal surgery, ensuring the needle is accurately inserted along the puncture path to the target point is crucial for surgical success. Figure 1The diagram illustrates a traditional biopsy. The standard procedure involves attaching a positioning net to the patient, taking anteroposterior and lateral X-ray images, and then marking the puncture point and direction on the skin based on the target point's position on the net. The puncture is then performed according to these markings. However, because the markings only indicate the top-view direction and not the side-view direction, there is no guidance during the puncture. Furthermore, after each puncture, another anteroposterior and lateral X-ray image is required to confirm correctness. Traditional biopsies are not precise and are unsafe.

[0039] Based on this, this application proposes an image processing method for medical assistance, applied to a C-arm machine. Compared to traditional C-arm machines, this application's C-arm machine adds a visible light camera. During surgery, it captures X-ray and visible light images, fuses them to obtain a fused image, and displays the fused image in real time on a screen. This allows medical personnel to intuitively observe the patient's physiological structures (e.g., the spine) and surgical equipment (e.g., puncture needles), providing medical assistance. For puncture surgeries, because the fused image allows for direct observation of whether the puncture needle has punctured the target area, there is no need to retake anteroposterior and lateral X-ray images after the puncture for confirmation, reducing radiation dose and extending machine lifespan.

[0040] Figure 2 This is a flowchart of an image processing method for medical assistance according to an embodiment of this application.

[0041] As an example, such as Figure 2 As shown, image processing methods for medical assistance include:

[0042] S201, acquires X-ray images based on the X-ray imaging module and acquires real-time visible light images based on the visible light camera, wherein the X-ray images and real-time visible light images are captured from the same angle, and both the X-ray images and real-time visible light images include images of the calibration tool.

[0043] S202, based on the position information of the calibration tool, matches and fuses the X-ray image and the real-time visible light image to obtain a real-time fused image, which is then displayed. The real-time fused image includes the target region. The target region is the area corresponding to the target image captured during the previous imaging process; this target can be a lesion. Of course, this target region can be marked in the fused image or it can be an area visible to the doctor.

[0044] S203, determines whether the surgical procedure has been completed based on the displayed real-time fused image and the target area.

[0045] For example, an image processing method for medical assistance is applied to a C-arm machine, which includes an X-ray imaging module and a visible light camera. Figure 3 The schematic diagram of the C-arm machine shown illustrates a C-arm with a C-shaped arm. One end of the C-arm has a radiation source, and the other end has a flat panel detector. The C-arm is rotatable and adjustable, allowing it to rotate around the patient to capture three-dimensional images, or to remain at any position along its rotational path to capture X-ray images. This application also includes a visible light camera mounted on the C-arm machine. This camera captures visible light images and can also acquire real-time visible light images. The visible light camera can capture images at preset time intervals, acquiring real-time visible light images from multiple consecutive frames. To facilitate the fusion of X-ray and visible light images, the visible light camera can be mounted near the flat panel detector at the top of the C-arm, next to the radiation source, or at a side column of the C-arm, ensuring that the shooting angles of the X-ray and real-time visible light images are consistent.

[0046] For example, such as Figure 3 The diagram shows the installation of the visible light cameras on a C-arm surgical machine. Visible light camera 1 is mounted to the flat panel detector, and visible light camera 2 is mounted on the side support of the C-arm. Whether visible light camera 1 is mounted on the flat panel detector or next to the radiation source can be set based on the initial positions of the flat panel detector and the radiation source. It's understood that the C-arm can be positioned differently; it's uncertain whether the flat panel detector or the radiation source is on top. Whichever is on top, that's where the visible light camera is installed, as the one below will be blocked by the operating table. Alternatively, visible light cameras can be installed in all positions, adaptively selecting the required camera based on the needs of the surgical scenario.

[0047] For example, a C-arm machine acquires real-time visible light images using a visible light camera and X-ray images using an X-ray imaging module. The machine also includes an image processing device that receives the real-time visible light images and X-ray images, and matches and fuses them to obtain a real-time fused image. To facilitate the matching of the real-time visible light images and X-ray images, a calibration tool can be placed in the imaging area when capturing the images. The calibration tool can be three-dimensional or two-dimensional. Two-dimensional calibration fixtures need to be moved between the two imaging positions, while three-dimensional fixtures can be placed directly on the patient. It should be noted that the calibration tool can be used during the initial fusion of X-ray and visible light images and may not be needed during subsequent surgery. Alternatively, it can be used to compare and determine the needle insertion position during puncture.

[0048] For example, both the X-ray image and the real-time visible light image obtained in this way include the image of the calibration tool. Based on the position information of the calibration tool, the X-ray image and the real-time visible light image are matched and fused to obtain a real-time fused image. The real-time fused image includes the target area. The image processing device in the C-arm machine is also connected to a display screen, which displays the real-time fused image. Because the visible light image is real-time, based on the displayed real-time fused image and the target area, the surgeon's instruments can be further displayed, providing a basis for the surgeon's judgment during the surgical procedure, thereby guiding the surgical operation. The target area can be understood as the patient's lesion area, for example, the puncture target area. The surgeon can observe the target area during the procedure, or the target can be identified and marked on the real-time fused image, and then displayed on the fused image.

[0049] The image processing method for medical assistance in this application combines X-ray images and visible light images, fuses the two to obtain a fused image, and displays the fused image on a display screen in real time, enabling medical personnel to intuitively observe the patient's physiological structure and the status of surgical equipment, thus providing medical assistance.

[0050] As an example, the X-ray image includes a first-angle X-ray image and a second-angle X-ray image, and the real-time visible light image includes a first-angle real-time visible light image and a second-angle real-time visible light image. At least one of the first angle and the second angle is adjustable. The X-ray image and the real-time visible light image are matched and fused based on the position information of the calibration tool to obtain a real-time fused image, which is then displayed, including:

[0051] Based on the position information of the calibration tool, the first angle X-ray image and the first angle real-time visible light image are matched and fused to obtain the first angle real-time fused image, and the first angle real-time fused image is displayed.

[0052] And / or, based on the position information of the calibration tool, the second-angle X-ray image and the second-angle real-time visible light image are matched and fused to obtain the second-angle real-time fused image, and the second-angle real-time fused image is displayed.

[0053] For example, to more comprehensively display the patient's physiological structure and surgical equipment, this application provides two shooting perspectives: a first angle and a second angle. At least one of the first and second angles is adjustable. For instance, the first angle is adjustable while the second angle is not. For example, if a visible light camera 1 is mounted on a C-arm, with one end of the C-arm next to the flat panel detector, the visible light camera 1 can adjust as the C-arm rotates. A visible light camera 2 is fixedly mounted... Figure 3In the rack position shown, the visible light camera 2 is fixed. In actual use, the position of the C-arm can be adjusted. The positions of the visible light camera 1 and the visible light camera 2 cannot be adjusted together. In this way, the relative positions of the visible light camera 1 and the visible light camera 2 are not fixed, which can realize the observation of the patient's symptom position from a set of relatively unfixed positions. The camera can be adjusted to select a more suitable angle for observing the symptom position.

[0054] For example, both the first and second angles can be adjustable, and can be relatively fixed or relatively non-fixed. For instance, visible light camera 1 is fixedly mounted on one end of the C-arm, i.e., on the flat panel detector, while visible light camera 2 is fixedly mounted on the C-arm at a preset angle relative to visible light camera 1. This preset angle is less than 180 degrees, preferably less than 100 degrees. When selecting an angle, the operator adjusts the rotation of the C-arm to choose the shooting angle of the two cameras relative to the patient, thereby enabling observation of the symptom location from an angle favorable to the operator. In practical use, the relative positions of visible light camera 1 and visible light camera 2 are fixed, but the position of the C-arm can be adjusted, and the positions of camera 1 and camera 2 can be adjusted together, allowing observation of the patient's symptom location from a set of relatively fixed positions.

[0055] For example, both visible light camera 1 and visible light camera 2 can be positioned adjustablely on the C-arm, or one can be adjustablely positioned on the frame while the other is fixedly mounted on the C-arm. This allows for greater and more flexible adjustment of the observation angle.

[0056] It is understandable that during surgical procedures, patients may not lie completely flat on the operating table due to their physiological structure or surgical needs. In this application, at least one of the first and second angles of the visible light camera is adjustable to facilitate medical staff in adjusting the camera position according to the actual situation and to more easily observe the location of the patient's condition.

[0057] For example, the adjustable range of the first angle includes an orthopedic position, while the second angle can be a fixed position. To ensure a comprehensive view of the patient's condition in real-time fused images, the first angle can be adjusted to an orthopedic position; for example, a top-down view of the patient lying supine on the operating table can be used as the orthopedic position. The second angle can be fixed to the lateral support column of the C-arm machine, using a lateral view as the second angle. The following example illustrates this with the first angle as orthopedic and the second angle as lateral. Figure 3 As shown, the shooting angle above the patient is the first angle, and the shooting angle to the right is the second angle.

[0058] For example, the X-ray image includes a first-angle X-ray image and a second-angle X-ray image, where the first angle can be an anterior view and the second angle can be a lateral view. Both the anterior and lateral X-ray images can be obtained using a C-arm X-ray machine. The real-time visible light image includes a first-angle real-time visible light image and a second-angle real-time visible light image. The anterior visible light image can be acquired by a visible light camera mounted on a flat panel detector or by a visible light camera mounted next to the radiation source. The lateral visible light image can be acquired by a visible light camera mounted on the lateral support of the C-arm X-ray machine. The shooting angle of the anterior X-ray image is the same as that of the anterior visible light image, and the shooting angle of the lateral X-ray image is the same as that of the lateral visible light image. Both the anterior and lateral X-ray images and the real-time visible light image include images of the calibration tool.

[0059] For example, matching and fusing X-ray images and real-time visible light images includes matching and fusing the anteroposterior X-ray image and the anteroposterior real-time visible light image based on the position information of the calibration tool to obtain an anteroposterior real-time fused image. And / or, matching and fusing the lateral X-ray image and the lateral real-time visible light image based on the position information of the calibration tool to obtain a lateral real-time fused image. Figure 4 The above-view visible light image and the above-view X-ray image are fused together to obtain a real-time fused above-view image. For example... Figure 5 The lateral visible light image and lateral X-ray image shown are fused to obtain a real-time lateral fused image. The fused image is transmitted to a display screen, where the anterior real-time fused image and the lateral real-time fused image are displayed, as shown. Figure 6 As shown.

[0060] This application provides multi-view fused images, allowing operators to directly observe the fused real-time patient image and perform surgery. With guidance from both top and side views, the accuracy is higher and the safety is enhanced.

[0061] As an example, the real-time fused images include images of the puncture needle, and the surgical procedure includes the puncture procedure, such as... Figure 7 As shown, surgical procedures guided by real-time fused images and target regions include:

[0062] S701 identifies the puncture needle in the real-time fused image and obtains the identification result of the puncture needle.

[0063] S702 guides the puncture procedure based on the real-time fused image, the target area, and the identification results of the puncture needle. Specifically, it can dynamically display the target area, the puncture needle, and the identification results of the puncture needle on the displayed fused image, providing doctors with reference and thus guiding the puncture.

[0064] For example, taking a puncture procedure as an example, when the puncture needle enters the imaging area to prepare for puncture, the real-time anteroposterior and lateral visible light images captured by the visible light camera contain the image of the puncture needle. Therefore, the real-time fused anteroposterior and lateral images also contain the image of the puncture needle. The puncture needle can be identified using a recognition algorithm in the real-time fused image, yielding the identification result. The identification result includes information such as the position of the puncture needle and its length in the image. The target area can be detected by a detection algorithm to identify the lesion area, or it can be observed by medical personnel. The puncture operation is guided based on the real-time fused image and the identification result of the puncture needle in the target area. For example, medical personnel can observe whether the puncture needle has reached the target area in the real-time fused image.

[0065] It should be noted that the image processing method for medical assistance described in this application is also applicable to other surgical situations requiring positioning by a C-arm machine, such as incision planning and cutting in open surgery, percutaneous stapler implantation, and percutaneous ultrasound therapy. The following explanation uses puncture surgery as an example.

[0066] As an example, the identification results of the puncture needle include the positional information of the target segment of the puncture needle outside the body and the positional information of the extension line of the puncture needle; such as Figure 8 As shown, the puncture needle is identified by performing real-time fused images, and the identification results of the puncture needle include:

[0067] S801 identifies the puncture needle in the real-time fused image and obtains the position information of the target segment of the puncture needle outside the body.

[0068] S802, obtain the position information of the extension line of the puncture needle based on the position information of the displayed target segment.

[0069] For example, a recognition algorithm is used to identify the target segment of the puncture needle outside the body from the real-time fused image, such as from anterior and lateral real-time fused images, or even from anterior and lateral visible light images. Based on the location information of the target segment of the puncture needle outside the body, such as the direction of puncture and the puncture point in contact with the patient, the location information of the extension line of the puncture needle is obtained. The location information of the extension line of the puncture needle can be seen passing through the patient's body to help medical personnel determine whether the extension line of the puncture needle crosses the target area. The puncture hole and its extension line position can be marked in both the anteroposterior and lateral real-time fused images.

[0070] This allows for puncture guidance from both anteroposterior and lateral views. During this process, medical personnel can adjust the position and direction of the puncture needle in physical space until the extended line is observed to pass through the puncture target area in real-time anteroposterior and lateral fusion images, providing medical assistance.

[0071] As an example, the identification results of the puncture needle include the location information of the target segment of the puncture needle outside the body, the location information of the extension line of the puncture needle, and the location information of the needle tip, such as... Figure 9 As shown, the puncture needle is identified by performing real-time fused images, and the identification results of the puncture needle include:

[0072] S901 identifies the puncture needle in the real-time fused image and obtains the position information of the target segment of the puncture needle outside the body.

[0073] S902 determines the insertion depth of the puncture needle based on the location information of the target segment.

[0074] S903 determines the needle tip position information based on the target segment's position information and the needle insertion depth.

[0075] For example, the location information of the target segment of the puncture needle outside the body is obtained by identifying the real-time fused image. This application can also calculate the length of the target segment of the puncture needle in the anteroposterior and lateral real-time fused images using algorithms. For example, a calibration tool can be placed in the image, and the length of the target segment of the puncture needle in the image can be identified based on the relative distance between the puncture needle and the calibration tool. For another example, the puncture needle may be composed of multiple segments, and the length of the target segment can be determined based on the number of segments in the target segment. The insertion depth of the puncture needle is determined by combining the total length of the puncture needle and the length of the target segment. The needle tip position information is determined based on the location information of the target segment and the insertion depth of the puncture needle. For example, the needle tip position information of the puncture needle inside the patient's body can be determined based on the puncture hole, its extension line, and the insertion depth. The real-time needle tip position can also be marked in the anteroposterior and lateral real-time fused images so that the operator can observe whether the needle tip position has reached the puncture target area.

[0076] As an example, the C-arm machine also includes a display screen, and the image processing method for medical assistance also includes: displaying the identification result of the puncture needle on the display screen and displaying a real-time fused image.

[0077] For example, the location of the puncture site and its extension line can be marked in both the anteroposterior and lateral real-time fused images. This allows for puncture guidance from both anteroposterior and lateral perspectives. During this process, medical personnel can adjust the position and direction of the puncture needle in physical space until the extension line is observed to pass through the puncture target area in both the real-time anteroposterior and lateral fused images.

[0078] For example, the real-time needle tip position can also be marked in the anteroposterior real-time fused image and the lateral real-time fused image so that the operator can observe whether the needle tip position has reached the puncture target area.

[0079] As an example, the puncture operation is guided based on the displayed real-time fused image, the target area, and the identification results of the puncture needle, including prompting to complete the puncture operation when the needle tip position information indicates that the needle tip position has reached the puncture target area.

[0080] For example, the image processing device can also detect the puncture target area according to a detection algorithm. Of course, the target area can also be observed by medical personnel. For instance, medical personnel can observe the puncture target area and mark it on the real-time fused image. The C-arm machine can identify the needle tip position information. If the needle tip position information indicates that the needle tip has reached the puncture target area, the system will prompt the completion of the puncture operation. For example, a pop-up prompt box can be used to remind medical personnel to complete the puncture operation.

[0081] As an example, the image processing method for medical assistance also includes issuing a prompt message when the distance between the needle tip and the puncture target area is less than a preset distance, wherein the prompt message indicates that the puncture needle is about to reach the puncture target area.

[0082] For example, this application can also issue a prompt message when the distance between the needle tip and the puncture target area is less than a preset distance, to remind medical personnel that the puncture needle is about to reach the puncture target area. The preset distance can be set in advance.

[0083] The image processing method for medical assistance described in this application allows the operator to directly observe the fused real-time frontal and lateral images during puncture. Because of the simultaneous guidance of top and side views, accuracy is higher and safety is enhanced. Alternatively, the display screen can simulate the puncture path to guide the doctor during the puncture, resulting in higher efficiency. After the puncture is completed, there is no need to take another X-ray frontal and lateral image for confirmation, reducing radiation dose.

[0084] This application also proposes a C-arm machine.

[0085] As an example, such as Figure 3As shown, the C-arm machine includes: an X-ray imaging module, which includes a flat panel detector and a radiation source disposed opposite to the flat panel detector, and is used to capture X-ray images; a C-arm, one end of which is connected to the flat panel detector and the other end of which is connected to the radiation source, and the C-arm is slidably mounted on a side column of the C-arm; at least two visible light cameras, which are mounted on the C-arm or the side column of the C-arm and are adapted to capture real-time visible light images from a preset angle corresponding to the X-ray images, and at least one of the at least two visible light cameras is position-adjustable; a processing device, which is connected to the X-ray imaging module and the visible light cameras, and is used to match and fuse the X-ray images and the real-time visible light images to generate a real-time fused image, which includes a target area, and guides surgical operations based on the displayed real-time fused image and the target area; and a display screen, which is connected to the processing device and is used to display the real-time fused image.

[0086] As an example, the number of visible light cameras is two. The first visible light camera is mounted at one end of the C-arm and is adapted to capture real-time visible light images from an adjusted angle, including the orthogonal position. The second visible light camera is fixed on the side column of the C-arm and is used to capture real-time visible light images from the side of the operating table.

[0087] For example, such as Figure 3 As shown, there can be two visible light cameras. The first visible light camera 1 is mounted on a C-shaped arm and positioned next to the flat panel detector. The second visible light camera 2 can be fixed on the C-shaped arm. In actual use, the position of the C-shaped arm can be adjusted, and the positions of camera 1 and camera 2 can be adjusted together. In this way, the relative positions of camera 1 and camera 2 are fixed, which allows observation of the patient's symptom location from a set of relatively fixed positions.

[0088] In another embodiment, the first visible light camera 1 is mounted on one side of the flat panel detector, and the second visible light camera 2 is fixed as follows. Figure 3As shown, a second visible light camera 2, fixed to the C-arm gantry, can capture images of the patient from one side of the operating table. A first visible light camera 1, fixed to the C-arm, can be used in practice. The positions of the C-arm and cameras 1 and 2 cannot be adjusted simultaneously, resulting in a non-fixed relative position between them. This allows for observation of the patient's lesion location from a set of relatively variable positions, and the camera angle can be adjusted to select a more suitable observation angle. Based on this, the first visible light camera 1 has an adjustable shooting angle (first angle). The second visible light camera 2, fixed to the C-arm gantry, captures images of the patient from the side of the operating table; its position remains constant during image capture and navigation, corresponding to a second angle. Based on this C-arm gantry, the aforementioned method steps can be implemented. The C-arm rotates, causing the X-ray imaging module to scan around the patient to acquire three-dimensional CT data, which includes the target area. The C-arm gantry captures CT data of the target area of ​​the patient. During this process, the X-ray imaging module of the C-arm rotates around the patient to capture images, and three-dimensional CT data can be obtained through reconstruction. This data can be displayed as a three-dimensional image on the screen.

[0089] The processing device is used to: project CT data from multiple positions rotated to by the C-arm scanning along the radial direction of the scanning trajectory to obtain X-ray images from multiple angles; identify targets based on the multiple X-ray images, obtain their corresponding target regions, and use the X-ray image corresponding to the largest target region as the first angle X-ray image; and project the CT data from the shooting angle corresponding to the second visible light camera to obtain the second angle X-ray image. The multiple positions rotated to by the C-arm scanning along the radial direction of the scanning trajectory are used to project the CT data. Here, the rotated positions and radial directions correspond to the CT data scanning and imaging process, and the corresponding angles are the angles from which the first visible light camera can capture the patient; obtain X-ray images from multiple angles; identify targets based on the multiple X-ray images, obtain their corresponding target regions; compare the target regions to determine the largest target region; use the shooting angle corresponding to the largest target region of the X-ray image as the first angle; and project the CT data from the shooting angle corresponding to the second visible light camera to obtain the second angle X-ray image. The steps of acquiring real-time visible light images using visible light cameras, specifically acquiring real-time visible light images from the same shooting angle as X-ray images, include: adjusting the first visible light camera 1 to the shooting angle corresponding to the largest target area of ​​the X-ray image; and acquiring real-time visible light images using the first visible light camera 1 and the second visible light camera 2. Specifically, the C-arm rotates to adjust the first visible light camera to the shooting angle corresponding to the largest target area of ​​the X-ray image; the first visible light camera captures a real-time visible light image at a first angle, and the second visible light camera captures a real-time visible light image at a second angle. It should be noted that the aforementioned visible light and X-ray imaging modules of the C-arm machine can acquire corresponding images, the processing module can process the images, and it can also be used to control the visible light and X-ray imaging modules to acquire corresponding images.

[0090] For example, during surgery, a C-arm can be used to capture X-ray images of the patient from multiple angles. Each X-ray image contains a target area. To display the target area as clearly as possible and facilitate operation by medical staff, the shooting angle corresponding to the largest target area in the X-ray image can be taken as the first angle. The position of the visible light camera is adjusted so that the shooting angle of the first visible light camera matches the first angle, thus capturing a visible light image at the first angle. A second visible light camera, for example, is fixed to the C-arm gantry. Based on the second visible light camera, a second visible light image is captured. The CT data is then projected based on the second angle to obtain the second-angle X-ray image. In this way, the first-angle X-ray image and the first-angle visible light image can be fused, and the second-angle X-ray image and the second-angle visible light image can also be fused.

[0091] This application also proposes a computer-readable storage medium.

[0092] In this embodiment, a computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, it implements the steps of the above-described image processing method for medical assistance.

[0093] Figure 10 A block diagram of an electronic device provided in an embodiment of this application.

[0094] This application provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described image processing method for medical assistance.

[0095] like Figure 10 As shown, for ease of understanding, embodiments of this application illustrate a specific electronic device.

[0096] Electronic devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0097] like Figure 10 As shown, the device includes a computing unit 1001, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded into a random access memory (RAM) 1003 from a storage unit 1008. The RAM 1003 may also store various programs and data required for the operation of the electronic device. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0098] Multiple components in the electronic device are connected to the I / O interface 1005. These components include: an input unit 1006, such as a keyboard or mouse; an output unit 1007, such as various types of displays or speakers; a storage unit 1008, such as a hard disk or optical disk; and a communication unit 1009, such as a network interface card (NIC), a modem, or a wireless transceiver. The communication unit 1009 allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0099] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods described above, such as image processing methods for medical assistance. For example, in some embodiments, the image processing methods for medical assistance may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, the image processing methods for medical assistance described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform an image processing method for medical assistance by any other suitable means (e.g., by means of firmware).

[0100] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequential list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this application, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0101] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0102] In the description of this application, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this application, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0103] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0104] Furthermore, the terms "first," "second," etc., used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance, or implicitly specifying the number of technical features indicated in this embodiment. Therefore, features defined with terms such as "first" and "second" in the embodiments of this application can explicitly or implicitly indicate that the embodiment includes at least one of those features. In the description of this application, the word "multiple" means at least two or more, such as two, three, four, etc., unless otherwise explicitly and specifically defined in the embodiments.

[0105] In this application, unless otherwise explicitly specified or limited in the embodiments, the terms "installation," "connection," "joining," and "fixing" appearing in the embodiments should be interpreted broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral part; it can also be a mechanical connection, an electrical connection, etc. Of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication between two components, or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific implementation.

[0106] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0107] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A C-arm machine, characterized in that, The C-arm machine includes: An X-ray imaging module, comprising a flat panel detector and a radiation source disposed opposite to the flat panel detector, is used to capture X-ray images; A C-shaped arm, one end of which is connected to the flat panel detector, and the other end of which is connected to the radiation source; the C-shaped arm is slidably mounted on a side column of the C-shaped arm; At least two visible light cameras are mounted on the C-arm or a side column of the C-arm, and are adapted to capture real-time visible light images from a preset angle corresponding to the X-ray image. The position of at least one of the at least two visible light cameras is adjustable. A processing device is connected to the X-ray imaging module and the visible light camera. The processing device is used to match and fuse the X-ray image and the real-time visible light image to generate a real-time fused image. A display screen, connected to the processing device, is used to display the real-time fused image; The at least two visible light cameras include a first visible light camera and a second visible light camera; The C-arm rotates, causing the X-ray imaging module to scan around the patient to obtain three-dimensional CT data, which includes the target. The processing device is further configured to: project the CT data along the radial direction of the scanning trajectory to multiple positions rotated to by the C-arm scanning to obtain X-ray images at multiple angles; identify targets based on multiple X-ray images to obtain their respective target regions, and take the X-ray image corresponding to the largest target region as the X-ray image at the first angle; rotate the C-arm to adjust the first visible light camera to the shooting angle corresponding to the X-ray image at the first angle. Based on the shooting angle corresponding to the second visible light camera, the CT data is projected to obtain an X-ray image at the second angle; The first visible light camera captures a real-time visible light image at the first angle, and the second visible light camera captures a real-time visible light image at the second angle.

2. The C-arm machine according to claim 1, characterized in that, The number of visible light cameras is 2. The first visible light camera is installed at one end of the C-arm and is adapted to capture real-time visible light images from an adjusted angle, including the orthogonal position. The second visible light camera is fixed on the side column of the C-arm and is used to capture real-time visible light images from the side of the operating table.

3. The C-arm machine according to claim 1, characterized in that, The X-ray image includes a first-angle X-ray image and a second-angle X-ray image, and the real-time visible light image includes a first-angle real-time visible light image and a second-angle real-time visible light image, wherein at least one of the first angle and the second angle is adjustable; the processing device is further configured to: Based on the position information of the calibration tool, the first angle X-ray image and the first angle real-time visible light image are matched and fused to obtain the first angle real-time fused image, and the first angle real-time fused image is displayed. And / or, based on the position information of the calibration tool, the second angle X-ray image and the second angle real-time visible light image are matched and fused to obtain a second angle real-time fused image, and the second angle real-time fused image is displayed; The real-time fused image includes a target region, and the processing device is also used to guide surgical operations based on the displayed real-time fused image and the target region.

4. The C-arm machine according to claim 3, characterized in that, The real-time fused image includes an image of the puncture needle, and the processing device is further used for: The puncture needle is identified in the real-time fused image to obtain the puncture needle identification result.

5. The C-arm machine according to claim 4, characterized in that, The identification result of the puncture needle includes the position information of the target segment of the puncture needle located outside the body and the position information of the extension line of the puncture needle; the processing device is also used for: The puncture needle is identified in the real-time fused image to obtain the position information of the target segment of the puncture needle outside the body; The position information of the extension line of the puncture needle is obtained based on the position information of the target segment.

6. The C-arm machine according to claim 4, characterized in that, The identification result of the puncture needle includes the position information of the target segment of the puncture needle outside the body, the position information of the extension line of the puncture needle, and the position information of the needle tip. The processing device is also used for: The puncture needle is identified in the real-time fused image to obtain the position information of the target segment of the puncture needle outside the body; The insertion depth of the puncture needle is determined based on the position information of the target segment; The needle tip position information is determined based on the position information of the target segment and the insertion depth of the puncture needle.

7. The C-arm machine according to any one of claims 5-6, characterized in that, The display screen is also used to display the identification result of the puncture needle.

8. The C-arm machine according to claim 6, characterized in that, The processing device is also used for: If the needle tip position information indicates that the needle tip has reached the target area, a prompt will appear indicating that the puncture operation is complete.

9. The C-arm machine according to claim 8, characterized in that, The processing device is also used for: If the distance between the needle tip and the target area is less than a preset distance, a prompt message is issued, wherein the prompt message indicates that the puncture needle is about to reach the target area.

Images