Image processing method for medical assistance, C-arm machine, electronic equipment and medium

Abstract

The invention discloses an image processing method for medical assistance, a C-arm machine, electronic equipment and a medium, and relates to the medical field, the method is applied to the C-arm machine, the C-arm machine comprises an X-ray shooting module and a visible light camera, the method comprises the following steps: obtaining an X-ray image based on the X-ray shooting module, and obtaining a real-time visible light image based on the visible light camera, the shooting angles of the X-ray image and the real-time visible light image are consistent, and each of the X-ray image and the real-time visible light image comprises an image of the calibration tool; matching and fusing the X-ray image and the real-time visible light image based on the position information of the calibration tool to obtain a real-time fused image, and displaying the real-time fused image; and guiding the surgical operation based on the displayed real-time fused image and the target area. According to the method, the X-ray image and the visible light image are combined to generate the fusion image in the operation, so that medical staff can visually observe the physiological structure of a patient and the condition of operation equipment through the fusion image, and medical assistance is provided.

Description

Technical Field

[0001] The present application relates to technical fields such as medical technology and medical assistance, and in particular to an image processing method, a C-arm machine, an electronic device and a medium for medical assistance. Background Art

[0002] In surgeries that use C-arms to provide positioning, such as percutaneous minimally invasive spinal surgery, allowing the puncture needle to accurately puncture the target point along the puncture path is the key to the success of the operation. The conventional operating steps are: first stick a positioning net on the human body, then take the frontal and lateral views, and the doctor marks the puncture point and puncture direction on the skin surface according to the position of the target point in the positioning net in the frontal and lateral views. Then perform the puncture operation based on the marked points and marking lines. However, since the marking lines on the skin surface can only indicate the direction of the top view and cannot give the direction of the side view, there is no guidance during the puncture process. In addition, after a puncture is completed, it is necessary to take the frontal and lateral X-ray images again to confirm whether the puncture is correct. Traditional puncture surgery is not accurate and is unsafe. Summary of the invention

[0003] To this end, the purpose of the implementation mode of the present application is to propose an image processing method, C-arm machine, electronic device, storage medium and computer program product for medical assistance, which combines X-ray images and visible light images to generate a fused image during surgery, so that medical personnel can intuitively observe the patient's physiological structure and the condition of surgical equipment through the fused image and provide medical assistance.

[0004] An embodiment of the present application provides an image processing method for medical assistance, the method is applied to a C-arm machine, the C-arm machine includes an X-ray shooting module and a visible light camera, the method includes: acquiring an X-ray image based on the X-ray shooting module, and acquiring a real-time visible light image based on the visible light camera, wherein the shooting angles of the X-ray image and the real-time visible light image are consistent, and the X-ray image and the real-time visible light image both include an image of a calibration tool; matching and fusing the X-ray image and the real-time visible light image based on position information of the calibration tool to obtain a real-time fused image, and displaying the real-time fused image, wherein the real-time fused image includes a target area; guiding surgical operations based on the displayed real-time fused image and the target area.

[0005] Exemplarily, the X-ray image includes a first-angle X-ray image and a second-angle X-ray image, the real-time visible light image includes a first-angle real-time visible light image and a second-angle real-time visible light image, and at least one of the first angle and the second angle is adjustable; the matching and fusing the X-ray image and the real-time visible light image based on the position information of the calibration tool to obtain a real-time fused image, and displaying the real-time fused image includes: matching and fusing the first-angle X-ray image and the first-angle real-time visible light image based on the position information of the calibration tool to obtain a first-angle real-time fused image, and displaying the first-angle real-time fused image; and / or matching and fusing the second-angle X-ray image and the second-angle real-time visible light image based on the position information of the calibration tool to obtain a second-angle real-time fused image, and displaying the second-angle real-time fused image.

[0006] Exemplarily, the real-time fused image includes an image of a puncture needle, the surgical operation includes a puncture operation, and guiding the surgical operation based on the displayed real-time fused image and the target area includes: identifying the puncture needle on the real-time fused image to obtain an identification result of the puncture needle; and guiding the puncture operation based on the displayed real-time fused image, the target area, and the identification result of the puncture needle.

[0007] Exemplarily, the recognition result of the puncture needle includes position information of a target segment of the puncture needle located outside the body and position information of an extension line of the puncture needle; the recognition of the puncture needle on the real-time fused image to obtain the recognition result of the puncture needle includes: recognizing the puncture needle on the real-time fused image to obtain position information of the target segment of the puncture needle located outside the body; and obtaining position information of the extension line of the puncture needle based on the position information of the target segment.

[0008] Exemplarily, the recognition result of the puncture needle includes position information of a target segment of the puncture needle located outside the body, position information of an extension line of the puncture needle, and position information of the needle tip. The recognition of the puncture needle on the real-time fused image to obtain the recognition result of the puncture needle includes: recognizing the puncture needle on the real-time fused image to obtain position information of the target segment of the puncture needle located outside the body; determining the insertion depth of the puncture needle based on the position information of the target segment; and determining the needle tip position information based on the position information of the target segment and the insertion depth of the puncture needle.

[0009] Exemplarily, the C-arm machine further includes a display screen, and the method further includes: displaying the recognition result of the puncture needle based on the display screen, and displaying the real-time fused image.

[0010] Exemplarily, guiding the puncture operation based on the displayed real-time fused image, the target area, and the recognition result of the puncture needle includes: when the needle tip position information indicates that the needle tip position has reached the target area, prompting to complete the puncture operation.

[0011] Exemplarily, the method further includes: when the distance between the needle tip position and the target area is less than a preset distance, issuing a prompt message, wherein the prompt message indicates that the puncture needle is about to reach the target area.

[0012] Another embodiment of the present application provides a C-arm, which includes: an X-ray shooting module, the X-ray shooting module includes a flat-panel detector and a radiation source arranged opposite to the flat-panel detector, the X-ray shooting module is used to shoot X-ray images; a C-arm, one end of the C-arm is connected to the flat-panel detector, and the other end of the C-arm is connected to the radiation source; the C-arm is slidably arranged on the C-arm lateral column; at least two visible light cameras, the at least two visible light cameras are arranged on the C-arm or the C-arm lateral column, and are suitable for shooting real-time visible light images from preset angles corresponding to the X-ray image, and at least one of the at least two visible light cameras is positionally adjustable; a processing device, the processing device is connected to the X-ray shooting module and the visible light camera, the processing device is used to match and fuse the X-ray image and the real-time visible light image to generate a real-time fused image, the real-time fused image includes a target area, and guides surgical operations based on the displayed real-time fused image and the target area; a display screen, the display screen is connected to the processing device, and the display screen is used to display the real-time fused image.

[0013] Exemplarily, the number of the visible light cameras is 2, the first visible light camera is installed at one end of the C-shaped arm, and is suitable for capturing real-time visible light images from an adjusted angle, wherein the adjusted angle includes a frontal position, and the second visible light camera is fixed on a lateral column of the C-shaped arm, and is used for capturing real-time visible light images from a lateral position of the operating bed.

[0014] Exemplarily, it is used to implement the steps of the above method;

[0015] The first visible light camera is installed at one end of the C-arm, rotates with the C-arm, and is set on one side of the flat-panel detector; the second visible light camera is fixed on the frame of the C-arm and is used to photograph the patient from one side of the operating table;

[0016] The C-shaped arm rotates to drive the X-ray shooting module to scan around the patient to obtain three-dimensional CT data, wherein the CT data includes the target;

[0017] The processing device is used for:

[0018] Based on the multiple positions to which the C-arm scans and rotates, the CT data is projected along the radial direction of the scanning trajectory to obtain X-ray images at multiple angles;

[0019] Identify the target based on the multiple X-ray images, obtain the target areas corresponding to each of them, and use the X-ray image corresponding to the largest target area as the X-ray image at the first angle;

[0020] Based on the shooting angle corresponding to the second visible light camera, the CT data is projected to obtain an X-ray image at a second angle;

[0021] The C-shaped arm rotates to adjust the first visible light camera to a shooting angle corresponding to the X-ray image corresponding to the largest target area;

[0022] The first visible light camera captures a real-time visible light image at the first angle, and the second visible light camera captures a real-time visible light image at the second angle.

[0023] Another embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method of any of the above embodiments when executing the computer program.

[0024] Another embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method of any of the above embodiments are implemented.

[0025] Another embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed by a processor of a computer device, the computer device is enabled to perform the steps of the method of any of the above embodiments.

[0026] In the above embodiment, the image processing method for medical assistance is applied to a C-arm machine, which includes an X-ray shooting module and a visible light camera, including: obtaining an X-ray image based on the X-ray shooting module, and obtaining a real-time visible light image based on the visible light camera, wherein the shooting angles of the X-ray image and the real-time visible light image are consistent, and both the X-ray image and the real-time visible light image include an image of a calibration tool; matching and fusing the X-ray image and the real-time visible light image based on the position information of the calibration tool to obtain a real-time fused image, and displaying the real-time fused image; guiding the surgical operation based on the displayed real-time fused image and the target area. The method of the present invention combines X-ray images and visible light images to generate a fused image during surgery, so that medical personnel can intuitively observe the patient's physiological structure and the condition of surgical equipment through the fused image to provide medical assistance. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1A schematic diagram of a traditional puncture operation provided for an embodiment of the present application;

[0028] Figure 2 A flowchart of an image processing method for medical assistance provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of a C-arm machine provided in an embodiment of the present application;

[0030] Figure 4 A schematic diagram of an orthotopic visible light image and an orthotopic X-ray image provided in an embodiment of the present application;

[0031] Figure 5 A schematic diagram of a lateral visible light image and a lateral X-ray image provided in an embodiment of the present application;

[0032] Figure 6 A schematic diagram of an anteroposterior real-time fusion image and a lateral real-time fusion image provided in an embodiment of the present application;

[0033] Figure 7 A flowchart for determining whether a surgical operation is completed based on real-time fusion images provided in an embodiment of the present application;

[0034] Figure 8 A flowchart for identifying a real-time fused image provided in an embodiment of the present application;

[0035] Fig. 9 A flowchart for identifying a real-time fused image provided by another embodiment of the present application;

[0036] Fig.10 A block diagram of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0037] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0038] In surgeries that use C-arms to provide positioning, such as percutaneous minimally invasive spinal surgery, allowing the puncture needle to accurately puncture the target point along the puncture path is the key to the success of the operation. Figure 1The schematic diagram of the traditional puncture surgery shown in the figure shows that the conventional operation steps are: first, a positioning net is attached to the human body, and then the frontal and lateral views are taken. The doctor marks the puncture point and direction on the skin surface according to the position of the target point in the positioning net in the frontal and lateral views. Then the puncture surgery is performed according to the marked points and marking lines. However, since the marking lines on the skin surface can only indicate the direction of the top view, and cannot give the direction of the side view, there is no guidance during the puncture process. In addition, after a puncture is completed, it is necessary to take the frontal and lateral X-ray images again to confirm whether the puncture is correct. Traditional puncture surgery is not accurate and is unsafe.

[0039] Based on this, the present application proposes an image processing method for medical assistance, which is applied to a C-arm machine. Compared with the traditional C-arm machine, the C-arm machine of the present application has an additional visible light camera. During the operation, X-ray images and visible light images are taken, and the two are fused to obtain a fused image. The fused image is displayed in real time on the display screen, so that medical personnel can intuitively observe the patient's physiological structure (such as the spine) and surgical equipment (such as puncture needles) to provide medical assistance. For puncture surgery, because the fused image can be used to intuitively observe whether the puncture needle has punctured the target area, there is no need to take the frontal and lateral X-ray images again for confirmation after the puncture, which reduces the radiation dose and extends the life of the machine.

[0040] Figure 2 It is a flowchart of an image processing method for medical assistance according to an embodiment of the present application.

[0041] As an example, Figure 2 As shown, the image processing method for medical assistance includes:

[0042] S201, acquiring an X-ray image based on an X-ray shooting module, and acquiring a real-time visible light image based on a visible light camera, wherein the shooting angles of the X-ray image and the real-time visible light image are consistent, and both the X-ray image and the real-time visible light image include an image of a calibration tool.

[0043] S202, based on the position information of the calibration tool, the X-ray image and the real-time visible light image are matched and fused to obtain a real-time fused image, and the real-time fused image is displayed, wherein the real-time fused image includes a target area. The target area is the target image captured in advance during the shooting, and is the area corresponding to the target, and the target may be a lesion. Of course, the target area may be marked in the fused image, or may be an area visible to the doctor.

[0044] S203: Determine whether the surgical operation is completed based on the displayed real-time fused image and the target area.

[0045] Exemplarily, the image processing method for medical assistance is applied to a C-arm machine, which includes an X-ray shooting module and a visible light camera. Figure 3 The schematic diagram of the C-arm machine shown in the figure comprises a C-arm, a radiation source is arranged at one end of the C-arm, and a flat-panel detector is arranged at the other end. The C-arm can be rotated and adjusted, and can rotate around the patient to take a three-dimensional image of the patient, or it can stay at any position on the rotation trajectory to take an X-ray image. The present application also installs a visible light camera on the C-arm machine, and the visible light camera is used to take visible light images and can also obtain real-time visible light images. The visible light camera can take pictures at preset time intervals and obtain real-time visible light images based on continuous multiple frames of visible light images. To facilitate the fusion of X-ray images and visible light images, the visible light camera can be installed near the flat-panel detector position on the top of the C-arm or next to the radiation source or at the side column position of the C-arm, so that the shooting angles of the X-ray image and the real-time visible light image can be consistent.

[0046] For example, Figure 3 The schematic diagram of the installation of the visible light camera of the C-arm is shown. The visible light camera 1 is installed on the flat-panel detector, and the visible light camera 2 is installed on the side column of the C-arm. Among them, whether the visible light camera 1 is installed on the flat-panel detector or next to the radiation source can be set to depend on the positions of the flat-panel detector and the radiation source in the initial position. It can be understood that the C-arm can be changed, and it is uncertain whether the flat-panel detector or the radiation source is on the top. The visible light camera is installed at whichever is on the top, because the one below will be blocked by the operating table. Of course, visible light cameras can also be installed in all positions, and the required visible light cameras can be adaptively called according to the needs of the surgical scene.

[0047] Exemplarily, a real-time visible light image is obtained based on a visible light camera, and an X-ray image is obtained based on an X-ray shooting module. The C-arm machine also includes an image processing device, which receives the real-time visible light image and the X-ray image, and matches and fuses the real-time visible light image and the X-ray image to obtain a real-time fused image. To facilitate the matching of the real-time visible light image and the X-ray image, a calibration tool can be placed in the shooting area when shooting the real-time visible light image and the X-ray image. The calibration tool can be three-dimensional or two-dimensional. When the two-dimensional calibration tool is used, it needs to be used at two shooting positions, and the three-dimensional one can be placed directly on the patient. It should be noted that the calibration tool can be used in the early stage of shooting the X-ray image and the visible light image for fusion, and it may no longer be used in the later surgery, or it can also be used for comparison to determine the needle insertion position during puncture.

[0048] Exemplarily, the X-ray image and the real-time visible light image taken in this way both include the image of the calibration tool, and the X-ray image and the real-time visible light image are matched and fused based on the position information of the calibration tool to obtain a real-time fused image. The real-time fused image includes the target area. The image processing device in the C-arm machine is also connected to the display screen, and displays the real-time fused image through the display screen. Since the visible light image is real-time, the doctor's operating tools can be further displayed based on the displayed real-time fused image and target area, providing a basis for judgment for the doctor to perform the surgical operation process, thereby guiding the surgical operation. The target area can be understood as the patient's lesion area, for example, the puncture target area. The doctor can observe the target area during the doctor's operation, or identify the target on the real-time fused image, mark the target area, and display the target area on the fused image.

[0049] The image processing method for medical assistance of the present application combines X-ray images and visible light images, fuses the two, obtains a fused image, and displays the fused image in real time on a display screen, so that medical personnel can intuitively observe the patient's physiological structure and the condition of surgical equipment, and provide medical assistance.

[0050] As an example, the X-ray image includes a first-angle X-ray image and a second-angle X-ray image, the real-time visible light image includes a first-angle real-time visible light image and a second-angle real-time visible light image, at least one of the first angle and the second angle is adjustable, the X-ray image and the real-time visible light image are matched and fused based on position information of the calibration tool to obtain a real-time fused image, and the real-time fused image is displayed, including:

[0051] Matching and fusing the first angle X-ray image and the first angle real-time visible light image based on the position information of the calibration tool to obtain a first angle real-time fused image, and displaying the first angle real-time fused image;

[0052] And / or, based on the position information of the calibration tool, the second angle X-ray image and the second angle real-time visible light image are matched and fused to obtain a second angle real-time fused image, and the second angle real-time fused image is displayed.

[0053] For example, in order to more comprehensively display the physiological structure of the patient and the situation of the surgical equipment, the present application provides two shooting angles, namely, the first angle and the second angle. Among them, at least one of the first angle and the second angle is adjustable, for example, the first angle is an adjustable position, and the second angle is not adjustable. For example, the visible light camera 1 is installed on the C-shaped arm, such as one end of the C-shaped arm is next to the flat panel detector, and the visible light camera 1 can be adjusted with the rotation of the C-shaped arm, and the visible light camera 2 is fixedly installed, such as Figure 3In the rack position shown, the position of the visible light camera 2 is fixed. During specific use, the position of the C-arm can be adjusted, and the positions of the visible light camera 1 and the visible light camera 2 cannot be adjusted together. In this way, the relative positions of the visible light camera 1 and the visible light camera 2 are not fixed, so that the patient's symptom position can be observed from a set of relatively non-fixed positions, and the camera can be adjusted at a more suitable angle for observing the symptom position.

[0054] Exemplarily, the first angle and the second angle may both be adjustable, and may be relatively fixed or relatively loose. For example, the visible light camera 1 is fixedly mounted on one end of the C-arm, i.e., on the flat-panel detector, and the visible light camera 2 is fixedly mounted on the C-arm at a preset angle relative to the visible light camera 1, wherein the preset angle is less than 180 degrees, preferably less than 100 degrees. When selecting the angle, the operator selects the shooting angle of the two cameras relative to the patient based on adjusting the rotation of the C-arm to a certain extent, thereby realizing the observation of the disease position from an angle that is convenient for the operator to observe. In specific use, the relative positions of the visible light camera 1 and the visible light camera 2 are fixed, and the position of the C-arm can be adjusted. The positions of the cameras 1 and 2 are adjusted together, so that the patient's disease position can be observed from a set of relatively fixed positions.

[0055] For another example, both the visible light camera 1 and the visible light camera 2 are adjustable in position on the C-shaped arm, or one is adjustable in position on the frame and the other is fixedly mounted on the C-shaped arm, so that the positions of both the visible light camera 1 and the visible light camera 2 can be adjusted, thereby achieving a greater degree of and more flexible adjustment of the observation angle.

[0056] It is understandable that during surgical operations, due to the patient's physiological structure or surgical requirements, the patient may not be lying completely flat on the operating table. In the present application, at least one of the first angle and the second angle of the position of the visible light camera is adjustable to facilitate medical staff to adjust the camera position according to actual conditions and to more conveniently observe the patient's disease location.

[0057] For example, the adjustable range of the first angle includes the anteroposterior position, and the second angle can be a fixed position. In order for the real-time fusion image to fully display the patient's condition, the first angle can be adjusted to the anteroposterior position. For example, the top view of the patient lying flat on the operating table can be used as the anteroposterior position, and the second angle can be a fixed position, which can be fixed to the lateral column of the C-arm machine, and the lateral view can be used as the second angle. The following example takes the first angle as the anteroposterior position and the second angle as the lateral position as an example. Figure 3 As shown, the shooting angle above the patient is the first angle, and the shooting angle on the right side is the second angle.

[0058] Exemplarily, the X-ray image includes a first-angle X-ray image and a second-angle X-ray image, the first angle may be an anteroposterior position, and the second angle may be a lateral position. Both the anteroposterior X-ray image and the lateral X-ray image may be obtained by taking pictures using a C-arm machine. The real-time visible light image includes a first-angle real-time visible light image and a second-angle real-time visible light image, wherein the anteroposterior visible light image may be obtained by a visible light camera installed on a flat-panel detector, or by a visible light camera installed next to the radiation source. The lateral visible light image may be obtained by a visible light camera installed at the lateral column position of the C-arm machine. The shooting angle of the anteroposterior X-ray image is consistent with the shooting angle of the anteroposterior visible light image, and the shooting angle of the lateral X-ray image is consistent with the shooting angle of the lateral visible light image. Both the anteroposterior and lateral X-ray images and the real-time visible light images contain images of the calibration tool.

[0059] Exemplarily, matching and fusing the X-ray image and the real-time visible light image includes matching and fusing the anteroposterior X-ray image and the anteroposterior real-time visible light image based on the position information of the calibration tool to obtain an anteroposterior real-time fused image. And / or matching and fusing the lateral X-ray image and the lateral real-time visible light image based on the position information of the calibration tool to obtain a lateral real-time fused image. Figure 4 The anteroposterior visible light image and anteroposterior X-ray image shown are fused to obtain an anteroposterior real-time fused image. Figure 5 The lateral visible light image and the lateral X-ray image shown are fused to obtain a lateral real-time fused image. The fused image is transmitted to a display screen, and the frontal real-time fused image and the lateral real-time fused image are displayed on the display screen, such as Figure 6 shown.

[0060] The present application provides multi-perspective fused images, allowing the operator to directly observe the real-time patient image after fusion and perform surgery. Due to the guidance of both the top view and the side view, the accuracy is higher and safer.

[0061] As an example, the real-time fused image includes an image of a puncture needle, and the surgical procedure includes a puncture operation, such as Figure 7 As shown, surgical procedures are guided based on the displayed real-time fused images and target areas, including:

[0062] S701, identifying the puncture needle on the real-time fused image to obtain a puncture needle identification result.

[0063] S702, guiding the puncture operation based on the displayed real-time fused image, the target area and the recognition result of the puncture needle. Specifically, the target area, the puncture needle and the recognition result of the puncture needle can be dynamically displayed on the displayed fused image to provide the doctor with reference, thereby guiding the puncture.

[0064] For example, taking the puncture surgery as an example, when the puncture needle enters the shooting area to prepare for puncture, the real-time frontal and lateral visible light images captured by the visible light camera contain the image of the puncture needle, so the real-time frontal and lateral fusion images also contain the image of the puncture needle. The puncture needle can be identified by the real-time fusion image through the recognition algorithm to obtain the recognition result of the puncture needle. The recognition result of the puncture needle includes, for example, the position of the puncture needle, the length of the puncture needle on the image and other information. The target area can be detected by the detection algorithm to obtain the lesion area, and can also be observed by medical staff. The puncture operation is guided based on the real-time fusion image and the recognition result of the puncture needle in the target area. For example, medical staff can observe whether the puncture needle has reached the target area on the real-time fusion image.

[0065] It should be noted that the image processing method for medical assistance of the present application is also applicable to other surgical occasions that require C-arm positioning, such as incision planning and cutting of open surgery, percutaneous nailing, percutaneous ultrasound treatment, etc. The following is an example of puncture surgery.

[0066] As an example, the recognition result of the puncture needle includes the position information of the target segment of the puncture needle outside the body and the position information of the extension line of the puncture needle; Figure 8 As shown, the puncture needle is identified on the real-time fusion image to obtain the puncture needle identification result, including:

[0067] S801, identifying the puncture needle on the real-time fusion image to obtain the position information of the target segment of the puncture needle located outside the body.

[0068] S802: Obtain the position information of the extension line of the puncture needle based on the displayed position information of the target segment.

[0069] Exemplarily, the real-time fusion image is recognized by a recognition algorithm, for example, from the real-time fusion image of the frontal and lateral positions, and of course, it can also be recognized based on the visible light image of the frontal and lateral positions, to obtain the position information of the target segment of the puncture needle located outside the body. According to the position information of the target segment of the puncture needle located outside the body, for example, the puncture direction of the puncture needle and the puncture position in contact with the patient. According to the position information of the target segment, the position information of the extension line of the puncture needle is obtained. The position information of the extension line of the puncture needle can pass through the patient's body to help medical personnel determine whether the extension line of the puncture needle passes through the target area. The puncture hole and its extension line position can be marked in the frontal real-time fusion image and the lateral real-time fusion image respectively.

[0070] In this way, the puncture can be guided from both the frontal and lateral directions. During this process, medical personnel can adjust the position and direction of the puncture needle in the physical space until the extension line is observed to pass through the target puncture area from the real-time frontal and lateral fusion images, thereby providing medical assistance.

[0071] As an example, the recognition result of the puncture needle includes the position information of the target segment of the puncture needle located outside the body, the position information of the extension line of the puncture needle, and the needle tip position information, such as Fig. 9 As shown, the puncture needle is identified on the real-time fusion image to obtain the puncture needle identification result, including:

[0072] S901, identifying the puncture needle on the real-time fusion image to obtain the position information of the target segment of the puncture needle located outside the body.

[0073] S902: Determine the insertion depth of the puncture needle based on the position information of the target segment.

[0074] S903, determining needle tip position information based on the target segment position information and the insertion depth of the puncture needle.

[0075] Exemplarily, the real-time fusion image is identified to obtain the position information of the target segment of the puncture needle located outside the body. The present application can also calculate the length of the target segment of the puncture needle in the real-time fusion image in the anteroposterior position and the real-time fusion image in the lateral position by an algorithm. For example, a calibration tool can be placed in the picture, and the length of the target segment of the puncture needle in the image can be assisted based on the relative distance between the puncture needle and the calibration tool. For example, the puncture needle can be composed of multiple sections, and the length of the target segment of the puncture needle can be determined based on the number of sections of the multiple sections of the puncture needle in the target segment. The insertion depth of the puncture needle is determined in combination with the total length of the puncture needle and the length of the target segment. The needle tip position information is determined based on the position information of the target segment and the insertion depth of the puncture needle. For example, the needle tip position information of the puncture needle located in the patient's body is determined based on the puncture hole, the extension line and the insertion depth. The real-time needle tip position can also be marked in the real-time fusion image in the anteroposterior position and the real-time fusion image in the lateral position so that the operator can observe whether the needle tip position reaches the puncture target area.

[0076] As an example, the C-arm machine also includes a display screen, and the image processing method for medical assistance also includes: displaying the recognition result of the puncture needle based on the display screen, and displaying the real-time fused image.

[0077] For example, the puncture hole and its extension line position can be marked in the real-time fusion image of the anteroposterior position and the real-time fusion image of the lateral position, respectively. In this way, the puncture can be guided from both the anteroposterior and lateral directions. In this process, the medical staff can adjust the position and direction of the puncture needle in the physical space until the extension line is observed to pass through the puncture target area from the real-time fusion image of the anteroposterior position and the fusion image of the lateral position.

[0078] For example, the real-time needle tip position may be marked in the real-time fusion image in the anteroposterior position and the real-time fusion image in the lateral position, so that the operator can observe whether the needle tip position reaches the puncture target area.

[0079] As an example, guiding the puncture operation based on the displayed real-time fused image, the target area and the recognition result of the puncture needle includes: prompting to complete the puncture operation when the needle tip position information indicates that the needle tip position reaches the puncture target area.

[0080] Exemplarily, the image processing device can also detect the puncture target area according to the detection algorithm. Of course, the target area can also be observed by medical personnel. For example, the medical personnel observe the puncture target area and mark the puncture target area on the real-time fusion image. The C-arm machine recognizes the needle tip position information. When the needle tip position information indicates that the needle tip position has reached the puncture target area, the puncture operation is completed. For example, a prompt box can pop up to prompt the medical personnel to complete the puncture operation.

[0081] As an example, the image processing method for medical assistance further includes: when the distance between the needle tip position and the puncture target area is less than a preset distance, issuing a prompt message, wherein the prompt message indicates that the puncture needle is about to reach the puncture target area.

[0082] For example, the present application can also issue a prompt message when the distance between the needle tip and the puncture target area is less than a preset distance to remind medical personnel that the puncture needle is about to reach the puncture target area. The preset distance can be set in advance.

[0083] The image processing method for medical assistance of the present application allows the operator to directly observe the real-time frontal and lateral images after fusion and perform puncture. Since there are both top and side views to guide, the accuracy is higher and safer. Alternatively, the puncture path can be simulated on the display screen to guide the doctor to perform the puncture, which makes the puncture more efficient. After the puncture is completed, there is no need to take X-ray frontal and lateral images again for confirmation, which reduces the radiation dose.

[0084] The application also proposes a C-arm machine.

[0085] As an example, Figure 3As shown, the C-arm machine includes: an X-ray shooting module, the X-ray shooting module includes a flat-panel detector and a radiation source arranged opposite to the flat-panel detector, and the X-ray shooting module is used to shoot X-ray images; a C-arm, one end of the C-arm is connected to the flat-panel detector, and the other end of the C-arm is connected to the radiation source, and the C-arm is slidably arranged on the C-arm lateral column; at least two visible light cameras, at least two visible light cameras are arranged on the C-arm or the C-arm lateral column, and are suitable for shooting real-time visible light images from preset angles corresponding to the X-ray images, and at least one of the at least two visible light cameras is positionally adjustable; a processing device, the processing device is connected to the X-ray shooting module and the visible light camera, and the processing device is used to match and fuse the X-ray image and the real-time visible light image to generate a real-time fused image, the real-time fused image includes a target area, and guides the surgical operation based on the displayed real-time fused image and the target area; a display screen, the display screen is connected to the processing device, and the display screen is used to display the real-time fused image.

[0086] As an example, the number of visible light cameras is 2. The first visible light camera is installed at one end of the C-arm and is suitable for taking real-time visible light images from an adjusted angle, wherein the adjusted angle includes the frontal position. The second visible light camera is fixed on the lateral column of the C-arm and is used to take real-time visible light images from the lateral position of the operating bed.

[0087] For example, Figure 3 As shown, the number of visible light cameras can be 2. The first visible light camera 1 is installed on the C-shaped arm and is arranged next to the flat panel detector. The second visible light camera 2 can be fixed on the C-shaped arm. In specific use, the position of the C-shaped arm can be adjusted, and the positions of the cameras 1 and 2 can be adjusted together. In this way, the relative positions of the cameras 1 and 2 are fixed, and the patient's symptom position can be observed from a set of relatively fixed positions.

[0088] In another embodiment, the first visible light camera 1 is mounted on one side of the flat panel detector, and the second visible light camera 2 is fixed as shown in FIG. Figure 3As shown, the second visible light camera 2 is fixed on the rack of the C-arm, and can shoot the patient from one side of the operating table. The first visible light camera 1 is fixed on the C-arm. In specific use, the position of the C-arm can be adjusted, and the positions of the camera 1 and the camera 2 cannot be adjusted together. In this way, the relative positions of the camera 1 and the camera 2 are not fixed, and the patient's symptom position can be observed from a group of relatively unfixed positions. The camera can be adjusted at an angle that is more suitable for observing the symptom position. Based on this, the shooting angle of the first visible light camera 1 is a first angle, and its position is adjustable. The second visible light camera 2 is fixed on the rack of the C-arm, and shoots the patient from the side of the operating table. During the shooting and navigation process, the position is unchanged, corresponding to the second angle. Based on the C-arm machine, the aforementioned method steps can be implemented. The C-arm rotates, driving the X-ray shooting module to scan around the patient to obtain three-dimensional CT data, and the CT data contains the target. The C-arm machine is used to shoot the CT data of the patient's target area. In this process, the X-ray shooting module of the C-arm machine rotates around the patient to shoot, and three-dimensional CT data can be obtained by reconstruction, and the data can be displayed as a three-dimensional image on the display screen.

[0089] The processing device is used to: project the CT data along the radial direction of the scanning trajectory based on the multiple positions to which the C-arm scan rotates, and obtain X-ray images at multiple angles; identify targets based on multiple X-ray images, obtain the corresponding target areas, and use the X-ray image corresponding to the largest target area as the X-ray image at the first angle; project the CT data based on the shooting angle corresponding to the second visible light camera to obtain the X-ray image at the second angle. Based on the multiple positions to which the C-arm scan rotates, project the CT data along the radial direction of the scanning trajectory, where the rotated positions and radial directions correspond to the scanning imaging process of the CT data, and the corresponding angle is the angle at which the first visible light camera can shoot the patient; obtain X-ray images at multiple angles, identify targets based on multiple X-ray images, obtain the corresponding target areas, compare the target areas, determine the largest target area, use the shooting angle corresponding to the X-ray image corresponding to the largest target area as the first angle, and project the CT data based on the shooting angle corresponding to the second visible light camera 2 to obtain the X-ray image at the second angle. The step of acquiring a real-time visible light image based on a visible light camera, that is, acquiring a real-time visible light image at the same shooting angle as an X-ray image, includes: adjusting the first visible light camera 1 to the shooting angle corresponding to the X-ray image corresponding to the largest target area, and acquiring the real-time visible light image using the first visible light camera 1 and the second visible light camera 2. Specifically, the C-arm rotates to adjust the first visible light camera to the shooting angle corresponding to the X-ray image corresponding to the largest target area; the first visible light camera captures a real-time visible light image at a first angle, and the second visible light camera captures a real-time visible light image at a second angle. It should be noted that the above-mentioned visible light and X-ray imaging modules based on the C-arm machine can acquire corresponding pictures, the processing module can process the images, and can also be used to control the visible light and X-ray imaging modules to acquire corresponding images, etc.

[0090] For example, during surgery, a C-arm can be used to take X-ray images at multiple angles around the patient. Each X-ray image has a target area. In order to display the target area as much as possible and facilitate the operation of medical staff, the shooting angle corresponding to the X-ray image corresponding to the largest target area can be used as the first angle, and the position of the visible light camera can be adjusted to make the shooting angle of the first visible light camera consistent with the first angle, and the visible light image of the first angle can be obtained. The second visible light camera is fixed on the frame of the C-arm, for example, and the visible light image of the second angle is obtained based on the second visible light camera. The CT data is projected according to the second angle to obtain the X-ray image of the second angle. In this way, the X-ray image of the first angle and the visible light image of the first angle can be fused, and the X-ray image of the second angle and the visible light image of the second angle can be fused.

[0091] The application also proposes a computer-readable storage medium.

[0092] In this embodiment, a computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned image processing method for medical assistance are implemented.

[0093] Fig.10 A block diagram of an electronic device provided for an embodiment of the present application.

[0094] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned image processing method for medical assistance when executing the computer program.

[0095] like Fig.10 As shown, for ease of understanding, the embodiment of the present application shows a specific electronic device.

[0096] Electronic devices are intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0097] like Fig.10 As shown, the device includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In RAM 1003, various programs and data required for the operation of the electronic device can also be stored. The computing unit 1001, ROM 1002, and RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0098] Multiple components in the electronic device are connected to the I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0099] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods described above, such as image processing methods for medical assistance. For example, in some embodiments, the image processing method for medical assistance may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 1002 and / or a communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, the image processing method for medical assistance described above may be executed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute the image processing method for medical assistance in any other appropriate manner (eg, by means of firmware).

[0100] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this application, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0101] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0102] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0103] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0104] In addition, the terms "first", "second", etc. used in the embodiments of the present application are only used for descriptive purposes and should not be understood as indicating or implying relative importance, or implicitly indicating the number of technical features indicated in the present embodiment. Therefore, the features defined by the terms "first", "second", etc. in the embodiments of the present application can explicitly or implicitly indicate that at least one of the features is included in the embodiment. In the description of the present application, the word "multiple" means at least two or two or more, such as two, three, four, etc., unless otherwise clearly and specifically defined in the embodiments.

[0105] In this application, unless otherwise clearly specified or limited in the embodiments, the terms "installed", "connected", "connected" and "fixed" etc. appearing in the embodiments should be understood in a broad sense. For example, the connection can be a fixed connection, a detachable connection, or an integrated connection. It can be understood that it can also be a mechanical connection, an electrical connection, etc.; of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements, or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to the specific implementation situation.

[0106] In the present application, unless otherwise clearly specified and limited, a first feature being “above” or “below” a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being “above”, “above”, and “above” a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being “below”, “below”, and “below” a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0107] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An image processing method for medical assistance, characterized in that: The method is applied to a C-arm machine, the C-arm machine includes an X-ray shooting module and a visible light camera, and the method includes: Acquire an X-ray image based on the X-ray shooting module, and acquire a real-time visible light image based on the visible light camera, wherein the X-ray image and the real-time visible light image have the same shooting angle, and both the X-ray image and the real-time visible light image include an image of a calibration tool; Matching and fusing the X-ray image and the real-time visible light image based on the position information of the calibration tool to obtain a real-time fused image, and displaying the real-time fused image, wherein the real-time fused image includes a target area; A surgical operation is guided based on the displayed real-time fused image and the target area.

2. The image processing method for medical assistance according to claim 1, characterized in that: The X-ray image includes a first-angle X-ray image and a second-angle X-ray image, and the real-time visible light image includes a first-angle real-time visible light image and a second-angle real-time visible light image, and at least one of the first angle and the second angle is adjustable; The matching and fusing of the X-ray image and the real-time visible light image based on the position information of the calibration tool to obtain a real-time fused image, and displaying the real-time fused image includes: Matching and fusing the first-angle X-ray image and the first-angle real-time visible light image based on the position information of the calibration tool to obtain a first-angle real-time fused image, and displaying the first-angle real-time fused image; And / or, based on the position information of the calibration tool, the second-angle X-ray image and the second-angle real-time visible light image are matched and fused to obtain a second-angle real-time fused image, and the second-angle real-time fused image is displayed.

3. The image processing method for medical assistance according to claim 1 or 2, characterized in that: The real-time fused image includes an image of a puncture needle, the surgical operation includes a puncture operation, and the surgical operation is guided based on the displayed real-time fused image and the target area, including: Identifying the puncture needle on the real-time fused image to obtain an identification result of the puncture needle; The puncture operation is guided based on the displayed real-time fused image, the target area and the recognition result of the puncture needle.

4. The image processing method for medical assistance according to claim 3, characterized in that: The recognition result of the puncture needle includes the position information of the target segment of the puncture needle located outside the body and the position information of the extension line of the puncture needle; the recognition of the puncture needle on the real-time fusion image to obtain the recognition result of the puncture needle includes: Identifying the puncture needle on the real-time fused image to obtain position information of a target segment of the puncture needle located outside the body; The position information of the extension line of the puncture needle is obtained based on the position information of the target segment.

5. The image processing method for medical assistance according to claim 3, characterized in that: The recognition result of the puncture needle includes the position information of the target segment of the puncture needle located outside the body, the position information of the extension line of the puncture needle and the needle tip position information. The recognition of the puncture needle on the real-time fused image to obtain the recognition result of the puncture needle includes: Identifying the puncture needle on the real-time fused image to obtain position information of a target segment of the puncture needle located outside the body; Determining the insertion depth of the puncture needle based on the position information of the target segment; The needle tip position information is determined based on the position information of the target segment and the insertion depth of the puncture needle.

6. The image processing method for medical assistance according to any one of claims 4-5, characterized in that: The C-arm machine further includes a display screen, and the method further includes: The recognition result of the puncture needle is displayed based on the display screen, and the real-time fused image is displayed.

7. The image processing method for medical assistance according to claim 5, characterized in that: The guiding of the puncture operation based on the displayed real-time fused image, the target area and the recognition result of the puncture needle comprises: When the needle tip position information indicates that the needle tip position has reached the target area, a prompt is provided to complete the puncture operation.

8. The image processing method for medical assistance according to claim 7, characterized in that: The method further comprises: When the distance between the needle tip position and the target area is less than a preset distance, a prompt message is issued, wherein the prompt message indicates that the puncture needle is about to reach the target area.

9. A C-arm machine, characterized in that: The C-arm machine comprises: An X-ray shooting module, the X-ray shooting module comprising a flat panel detector and a radiation source arranged opposite to the flat panel detector, the X-ray shooting module being used to shoot X-ray images; A C-shaped arm, one end of which is connected to the flat panel detector, and the other end of which is connected to the radiation source; the C-shaped arm is slidably arranged on a lateral column of the C-shaped arm; At least two visible light cameras, the at least two visible light cameras being arranged on the C-shaped arm or the side column of the C-shaped arm, and being adapted to capture a real-time visible light image from a preset angle corresponding to the X-ray image, and at least one of the at least two visible light cameras being positionally adjustable; a processing device, the processing device being connected to the X-ray shooting module and the visible light camera, the processing device being used to match and fuse the X-ray image and the real-time visible light image to generate a real-time fused image, the real-time fused image including the target area, and guiding the surgical operation based on the displayed real-time fused image and the target area; A display screen is connected to the processing device, and is used to display the real-time fused image.

10. The C-arm machine according to claim 9, characterized in that: The number of the visible light cameras is 2, the first visible light camera is installed at one end of the C-shaped arm, and is suitable for capturing real-time visible light images from an adjusted angle, wherein the adjusted angle includes a frontal position, and the second visible light camera is fixed on a lateral column of the C-shaped arm, and is used for capturing real-time visible light images from a lateral position of the operating bed.

11. The C-arm machine according to claim 10, characterized in that: The first visible light camera is installed at one end of the C-arm, rotates with the C-arm, and is set on one side of the flat-panel detector; the second visible light camera is fixed on the frame of the C-arm and is used to photograph the patient from one side of the operating table; The C-shaped arm rotates to drive the X-ray shooting module to scan around the patient to obtain three-dimensional CT data, wherein the CT data includes the target; The processing device is used for: Based on the multiple positions to which the C-arm scans and rotates, the CT data is projected along the radial direction of the scanning trajectory to obtain X-ray images at multiple angles; Identify the target based on the multiple X-ray images, obtain the target areas corresponding to each of them, and use the X-ray image corresponding to the largest target area as the X-ray image at the first angle; Based on the shooting angle corresponding to the second visible light camera, the CT data is projected to obtain an X-ray image at a second angle; The C-shaped arm rotates to adjust the first visible light camera to a shooting angle corresponding to the X-ray image corresponding to the largest target area; The first visible light camera captures a real-time visible light image at the first angle, and the second visible light camera captures a real-time visible light image at the second angle.

12. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of the method described in any one of claims 1 to 8 when executing the computer program.

Images