Image processing method and puncture guidance system

Abstract

The present application relates to an image processing method and a puncture guidance system. The method includes: acquiring a medical image sequence image and an ultrasound image sequence of a target object; performing registration based on the ultrasound image sequence and the medical image sequence image to obtain a registration transformation matrix and a three-dimensional image after registration fusion; displaying the cross-sectional image in the three-dimensional image after registration fusion, the mapping point of the puncture point on the puncture plate projected on the cross-sectional image, and the imaging area of ​​the sagittal plane in the first display area; in response to the rotation operation of the ultrasound probe, updating the cross-sectional image, the mapping point, and the imaging area of ​​the sagittal plane in the first display area in real time according to the rotation operation; when any target mapping point is located in the imaging area of ​​the sagittal plane in the cross-sectional image, the relative position of the target mapping point located in the imaging area of ​​the sagittal plane in the sagittal plane image is displayed in the second display area. In this way, the same target mapping point can be displayed simultaneously in the cross-sectional image and the sagittal plane image.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to an image processing method and a puncture guidance system. Background Art

[0002] In the medical field, the existing puncture guidance of lesions in organs and tissues is mostly through the use of ultrasound imaging with a stepper and a puncture plate. However, due to the characteristics of ultrasound imaging, the relative position of the convex array acoustic window of the ultrasound probe and the puncture plate must be fixed; therefore, when the puncture target is not in the sagittal plane, the projection position of the puncture target in the sagittal plane cannot be determined, and the ultrasound probe needs to be moved repeatedly to observe the puncture target. Generally, the spatial position relationship between the two planes is judged by identifying the key structural areas in the image. The steps are cumbersome, time-consuming and labor-intensive, and the accuracy and stability are relatively poor.

[0003] Therefore, how to display the same puncture target in both the cross-sectional image and the sagittal image at the same time, so as to accurately determine the spatial position relationship between the two planes, is an urgent problem to be solved. Summary of the invention

[0004] Based on this, it is necessary to provide an image processing method and a puncture guidance system to address the above technical problems.

[0005] In a first aspect, the present application provides an image processing method, which is applied to a puncture guidance system, and the method comprises:

[0006] Acquire a sequence of medical image images of the target object; wherein the sequence of medical image images includes a lesion area, an area to be ablated, and a contour of the target object;

[0007] Acquiring a sequence of ultrasonic images of the target object in real time through an ultrasonic probe;

[0008] Registering the ultrasound sequence images with the medical image sequence images to obtain a registration transformation matrix and a three-dimensional image after registration fusion; the registration transformation matrix is ​​used to indicate the spatial conversion relationship between the two-dimensional coordinate system corresponding to the ultrasound sequence images and the three-dimensional coordinate system corresponding to the medical image sequence images;

[0009] Displaying in the first display area the cross-sectional image in the registered and fused three-dimensional image, the mapping point where the puncture point on the puncture plate is projected on the cross-sectional image, and the imaging area of ​​the sagittal plane;

[0010] In response to the rotation operation of the ultrasound probe, updating the cross-sectional image, the mapping points, and the imaging area of ​​the sagittal plane in the first display area in real time according to the rotation operation;

[0011] When any target mapping point is located in the imaging area of ​​the sagittal plane in the cross-sectional image, the second display area displays the relative position of the target mapping point located in the imaging area of ​​the sagittal plane in the sagittal plane image; the target mapping point indicates the mapping point closest to the puncture target point; the puncture target point is located in the contour of the area to be ablated.

[0012] In one embodiment, the method further comprises:

[0013] When the ultrasonic probe and the puncture plate are orthogonal, the sensor coordinates, the first matrix and the second matrix of the positioning sensor in the corresponding space coordinate system are obtained through the positioning sensor connected to the ultrasonic probe; the first matrix is ​​a matrix determined according to the sensor coordinate transformation; the second matrix is ​​used to represent the first position transformation relationship between the positioning sensor and the ultrasonic probe;

[0014] determining a second position conversion relationship between the two-dimensional coordinate system and a spatial coordinate system corresponding to the positioning sensor according to the sensor coordinates, the first matrix and / or the second matrix;

[0015] When any one of the target mapping points is located in the imaging area of ​​the sagittal plane in the cross-sectional image, displaying in the second display area the relative position of the target mapping point located in the imaging area of ​​the sagittal plane in the sagittal plane image, comprises:

[0016] Determining coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the two-dimensional coordinate system according to the second position conversion relationship;

[0017] Determining target coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the three-dimensional coordinate system according to the coordinate data and the registration transformation matrix;

[0018] The relative position of the target mapping point located on the imaging area of ​​the sagittal plane in the sagittal plane image is displayed in the second display area according to the target coordinate data.

[0019] In one embodiment, determining the second position conversion relationship between the two-dimensional coordinate system and the spatial coordinate system corresponding to the positioning sensor according to the sensor coordinates, the first matrix and / or the second matrix includes at least one of the following:

[0020] The first method: obtaining the first coordinate of the target mapping point in the two-dimensional coordinate system, and determining the second coordinate of the target mapping point in the spatial coordinate system according to the first matrix, the second matrix and the first coordinate; establishing the second position conversion relationship according to the first coordinate and the second coordinate;

[0021] The second method: obtaining the vertex coordinates of the vertices of the cross-sectional image in the spatial coordinate system according to the first matrix, the second matrix and the size of the cross-sectional image; determining the second coordinates of the target mapping point in the spatial coordinate system according to the first coordinates and the vertex coordinates; establishing the second position conversion relationship according to the first coordinates and the second coordinates.

[0022] In one embodiment, determining the coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the two-dimensional coordinate system according to the second position conversion relationship includes:

[0023] Updating the second matrix of the positioning sensor in the spatial coordinate system to a third matrix; the third matrix is ​​used to represent a third position conversion relationship between the positioning sensor and the rotated ultrasound probe;

[0024] According to the second coordinates of the target mapping point located on the imaging area of ​​the sagittal plane and the third matrix, the coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the two-dimensional coordinate system corresponding to the updated cross-sectional image is determined using the second position transformation relationship.

[0025] In a second aspect, the present application also provides a puncture guidance system, including: an ultrasound probe and a control unit;

[0026] The ultrasound probe is communicatively connected with the control unit and is used to send ultrasound sequence images of the target object to the control unit in real time;

[0027] The control unit comprises:

[0028] An acquisition module, used to acquire a sequence of medical image images and the ultrasound sequence images of the target object; wherein the sequence of medical image images includes a lesion area, an area to be ablated, and a contour of the target object;

[0029] A registration module, used for performing registration based on the ultrasound sequence images and the medical image sequence images to obtain a registration transformation matrix and a three-dimensional image after registration fusion; the registration transformation matrix is ​​used to indicate the spatial conversion relationship between the two-dimensional coordinate system corresponding to the ultrasound sequence images and the three-dimensional coordinate system corresponding to the medical image sequence images;

[0030] A first processing module, used for displaying in a first display area the cross-sectional image in the registered and fused three-dimensional image, the relative position of the mapping point of the puncture point on the puncture plate projected on the cross-sectional image and the imaging area of ​​the sagittal plane;

[0031] The first processing module is further configured to respond to a rotation operation of the ultrasound probe and update the cross-sectional image, the mapping points and the imaging area of ​​the sagittal plane in the first display area in real time according to the rotation operation;

[0032] The second processing module is used to display the relative position of the target mapping point located in the imaging area of ​​the sagittal plane in the sagittal plane image in the second display area when any target mapping point is located in the imaging area of ​​the sagittal plane in the cross-sectional image; the target mapping point indicates the mapping point closest to the puncture target point; the puncture target point is located in the outline of the area to be ablated.

[0033] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the image processing method described in any embodiment of the present application is implemented.

[0034] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the image processing method described in any embodiment of the present application is implemented.

[0035] In the above-mentioned image processing method, in response to the rotation operation of the dual-plane ultrasound probe, during the rotation process of the dual-plane ultrasound probe, information such as the cross-sectional image, mapping points and the imaging area of ​​the sagittal plane can be updated in real time. When any target mapping point coincides with the imaging area of ​​the sagittal plane, the relative position of the target mapping point located on the imaging area of ​​the sagittal plane in the sagittal plane image can be displayed in the second display area; thereby, the same target mapping point can be displayed simultaneously in the cross-sectional image and the sagittal plane image. On the one hand, the puncture position of each puncture can be accurately determined, with relatively high accuracy and stability, reducing the occurrence of multiple punctures due to inaccurate puncture positions and improving patient experience; on the other hand, each puncture does not need to repeatedly move the dual-plane ultrasound probe to observe the puncture position. After the last puncture, the ultrasound probe can be directly rotated to the next target mapping point for puncture, simplifying the operation, saving time and improving puncture efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flowchart of an image processing method according to an exemplary embodiment;

[0037] Figure 2is a schematic diagram of a cross-sectional image according to an exemplary embodiment;

[0038] Figure 3 is a schematic diagram showing a lesion and a tumor margin according to an exemplary embodiment;

[0039] Figure 4 is a schematic diagram of a prostate organ according to an exemplary embodiment;

[0040] Figure 5 is a schematic diagram of a cross-sectional image after the ultrasound probe is rotated according to an exemplary embodiment;

[0041] Figure 6 is a schematic diagram showing a cross-sectional image and a sagittal image according to an exemplary embodiment;

[0042] Figure 7 is a schematic diagram showing a puncture plate and an ultrasound probe being orthogonal to each other according to an exemplary embodiment;

[0043] Figure 8 is a schematic diagram showing a display of an ultrasound probe after rotation according to an exemplary embodiment;

[0044] Fig. 9 is a schematic diagram of vertices of a cross-sectional image according to an exemplary embodiment;

[0045] Fig.10 is a flowchart of an image processing method according to an exemplary embodiment;

[0046] Fig.11 is a structural block diagram of a puncture guidance system according to an exemplary embodiment;

[0047] Fig.12 is a schematic diagram of a puncture plate according to an exemplary embodiment;

[0048] Fig.13 The figure is a diagram showing the internal structure of a computer device according to an exemplary embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] The terms "first", "second", "third" in the embodiments of the present application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second", "third" can expressly or implicitly include at least one of the features. In the description of the present application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0051] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0052] The image processing method in the embodiment of the present application can be applied to a computer device; the computer device can be any mobile terminal or fixed terminal. The terminal can be a device that provides voice and / or data connectivity to a user. Exemplarily, the terminal can be an Internet of Things terminal, such as a sensor device, a mobile phone or a so-called "cellular" phone and a computer with an Internet of Things terminal, for example, a fixed, portable, pocket-sized, handheld, or computer-built-in device.

[0053] The image processing method in the embodiment of the present application can be applied to applications of computer devices; the applications can be but are not limited to various software programs, quick applications and / or applets.

[0054] In the embodiment of the present application, at least one is used to indicate one or more; and multiple is used to indicate two or more.

[0055] In one embodiment, Figure 1 As shown, an image processing method is provided, and the image processing method comprises the following steps:

[0056] S101, acquiring a sequence of medical image images of a target object; wherein the sequence of medical image images includes a lesion region, a region to be ablated, and a contour of the target object.

[0057] In the embodiment of the present application, the medical imaging sequence images may include but are not limited to X-ray imaging, computerized tomography (CT) imaging and nuclear magnetic resonance imaging (MRI) imaging.

[0058] In the embodiment of the present application, the lesion indicates the part of the body where the lesion occurs. The lesion area indicates the area where the lesion is located in the medical image sequence.

[0059] Exemplarily, the lesion region can be obtained by inputting the medical image sequence image into a segmentation model (e.g., a convolutional neural network model) to perform lesion region segmentation. Alternatively, the lesion region can also be a manually outlined lesion region in the medical image sequence image. The division method of the lesion region is not limited here.

[0060] In the embodiment of the present application, the region to be ablated is used to indicate the region where abnormal cells are targeted for ablation; the area of ​​the region to be ablated is greater than or equal to the area of ​​the lesion region. Figure 2 as shown in .

[0061] Optionally, the region to be ablated may be a region that surrounds the lesion region in space and is generated as the region to be ablated based on the recognition result of the lesion region. Figure 3 As shown, the computer device inputs the multi-parameter magnetic resonance data of the prostate organ, uses the convolutional neural network to segment the prostate organ, and then outlines the lesion on the prostate organ; and based on the outline of the lesion, extends the preset distance outward along the normal direction as the tumor margin outline, and limits the tumor margin outline to not exceed the capsule range, and determines the tumor margin outline as the area to be ablated. For example, the preset distance can be 8mm, 10mm or 11mm, etc. Figure 4 As shown, Figure 4 It is a schematic diagram of the prostate organ, which includes the tumor margin and lesion.

[0062] S102, acquiring ultrasound sequence images of the target object in real time through an ultrasound probe.

[0063] In the embodiment of the present application, the ultrasound probe is used to obtain images of the cross section (cross section) and / or sagittal section (longitudinal section) and display them on the screen in real time. The ultrasound probe may include but is not limited to a single-plane ultrasound probe and a dual-plane ultrasound probe.

[0064] Illustratively, the ultrasound probe may be a transrectal biplane intracavitary probe.

[0065] In the embodiment of the present application, the transverse plane (tra), also known as the horizontal plane or the axial plane, is used to indicate the section parallel to the horizontal plane and perpendicular to the vertical axis when the human body is standing upright. In medical imaging, the transverse image is obtained by collecting data parallel to the transverse plane during scanning imaging.

[0066] In the embodiments of the present application, the sagittal plane (sag), also called the longitudinal section or lateral section, is used to indicate the section passing through the sagittal axis and perpendicular to the coronal plane and the horizontal plane. The sagittal plane divides the human body into two symmetrical parts. In medical imaging, the sagittal plane image corresponds to the longitudinal section along the front-to-back direction of the human body, which can show the symmetrical structures on both sides of the body and their relative positional relationship.

[0067] In some embodiments, when the ultrasound probe is a dual-plane ultrasound probe, the computer device can transmit high-frequency sound waves through the ultrasound probe. When these sound waves propagate in human tissues and encounter different interfaces (such as tissues, organs, blood vessels, etc.), they will produce reflection, refraction, and scattering phenomena. The ultrasound probe receives these reflected sound wave signals and converts them into electrical signals. After computer processing, images in two different directions are obtained; that is, the cross-sectional image and sagittal image of the target object are obtained simultaneously.

[0068] In some embodiments, when the ultrasound probe is a single-plane ultrasound probe, multiple cross-sectional images of the target object can be acquired, and all the cross-sectional images are superimposed. The superimposed cross-sectional images are reconstructed through multi-plane reconstruction technology to obtain a three-dimensional model / image of the target object, and a sagittal image is obtained by cutting from the three-dimensional model / image.

[0069] S103, registering the ultrasound sequence images with the medical image sequence images to obtain a registration transformation matrix and a three-dimensional image after registration fusion; the registration transformation matrix is ​​used to indicate the spatial conversion relationship between the two-dimensional coordinate system corresponding to the ultrasound sequence images and the three-dimensional coordinate system corresponding to the medical image sequence images.

[0070] In the embodiment of the present application, registration refers to the process of aligning different point cloud data of two or more images to the same coordinate system through steps such as feature extraction, initial registration and fine registration, so that they are consistent in spatial position. The registration method may include but is not limited to at least one of iterative closest point (ICP) registration, normal distribution transform (NDT) registration and random sample consensus (RANSAC) registration.

[0071] In some embodiments, the medical image sequence images may be nuclear magnetic resonance image sequence images; the computer device may register and overlap the ultrasound sequence images and the nuclear magnetic resonance image sequence, that is, overlap and display the two images on the same display area to achieve fusion display of different images and obtain a fused three-dimensional image, so that three-dimensional organs, tissues, areas to be ablated and lesion areas can be displayed in the three-dimensional image. In this way, since the ultrasound sequence images and the nuclear magnetic resonance image sequence images have been registered and fused, after the ultrasound probe is rotated, the updated cross-sectional images and sagittal images can be synchronously and accurately displayed on the corresponding fused three-dimensional image.

[0072] S104, displaying in a first display area the cross-sectional image in the registered and fused three-dimensional image, the relative position of the mapping point of the puncture point on the puncture plate projected on the cross-sectional image, and the imaging area of ​​the sagittal plane.

[0073] In the embodiment of the present application, the puncture plate can be a positioning guide plate for assisting puncture surgery. The puncture plate has a plurality of puncture points (puncture holes) arranged at fixed intervals.

[0074] In some embodiments, the computer device can project the puncture target point on the puncture plate onto the cross-sectional image according to the mapping relationship between the cross-sectional image and the puncture plate; the mapping relationship is used to indicate the mapping relationship between the coordinate data of the cross-sectional image and the coordinate data of the puncture point on the puncture plate.

[0075] In some embodiments, the computer device projects the puncture points of the puncture plate onto the cross-sectional image, and can determine the mapping points located in the area to be ablated; the puncture points on the puncture plate correspond one to one with the mapping points in the cross-sectional image. For example, when the ultrasound probe and the puncture plate are orthogonal, the schematic diagram of the projected cross-sectional image is as follows: Figure 2 shown. Figure 2 The mid-sectional image includes the anterior fibromuscular stroma (AS), central zone (CZ), anterior peripheral zone (PZa), posterior peripheral zone (PZpl), anterior transitional zone (TZa), posterior transitional zone (TZp), and area to be ablated (P_tumor) of the target object.

[0076] S105 , in response to the rotation operation of the ultrasound probe, updating the cross-sectional image, the mapping points, and the imaging area of ​​the sagittal plane in the first display area in real time according to the rotation operation.

[0077] In some embodiments, after the ultrasound probe rotates, the cross-sectional image is acquired and updated in real time, and the updated cross-sectional image, mapping points, and sagittal imaging area are displayed in the first display area; the computer device determines the mapping point closest to the puncture target point as the target mapping point. The puncture target point indicates the target puncture point of the area to be ablated on the target object determined by the computer device before registration; the target mapping point is used to guide the interventional ablation device to perform puncture and / or ablation.

[0078] In some embodiments, the imaging region representing the sagittal plane in the cross-sectional image is a centerline (symmetry axis) region in the cross-sectional image. Figure 2 The middle area to be ablated includes 6 target mapping points, and none of the target mapping points is located on the centerline area.

[0079] In some embodiments, when the target mapping point of the current puncture is not located in the imaging area of ​​the sagittal plane in the first display area, the computer device rotates the ultrasound probe until the target mapping point of the current puncture is located in the imaging area of ​​the sagittal plane. During the rotation process, the computer device can scan and display the first display area in real time through the ultrasound probe, and the first display area updates the cross-sectional image, the mapping point of the puncture target, and the imaging area of ​​the sagittal plane in real time according to the rotation operation.

[0080] For example, Figure 2 As shown, Figure 2 is a cross-sectional image before the ultrasound probe is rotated. Assuming that the target mapping point of this puncture is E3, E3 is located on the right side of the imaging area of ​​the sagittal plane in the cross-sectional image; Figure 5 As shown, Figure 5 It is a schematic diagram of a cross-sectional image after the ultrasound probe is rotated. At this time, the ultrasound probe and the puncture plate are not orthogonal, and E3 is located in the imaging area of ​​the sagittal plane in the cross-sectional image. Figure 5 The mid-sectional image includes the anterior fibromuscular stroma (AS), central zone (CZ), anterior peripheral zone (PZa), posterior peripheral zone (PZpl), anterior transitional zone (TZa), posterior transitional zone (TZp), and area to be ablated (P_tumor) of the target object.

[0081] S106, when any target mapping point is located in the imaging area of ​​the sagittal plane in the cross-sectional image, the relative position of the target mapping point located in the imaging area of ​​the sagittal plane in the sagittal plane image is displayed in the second display area; the target mapping point indicates the mapping point closest to the puncture target point; the puncture target point is located in the outline of the area to be ablated.

[0082] In some embodiments, the target object is the prostate gland, such as Figure 6 As shown, Figure 6 Schematic diagram of cross-sectional and sagittal images of the prostate. Figure 5The device includes a first display area and a second display area, wherein the first display area is used to display a cross-sectional image, and the second display area is used to display a sagittal image. The cross-sectional image in the first display area includes the anterior fibromuscular layer (AFS), transition zone (TZ), central zone (CZ) and ejaculatory duct (EJD) of the prostate; the sagittal image in the second display area includes the anterior fibromuscular layer (AFS) and peripheral zone (PZ) of the prostate. When any target mapping point is located in the imaging area of ​​the sagittal plane in the first display area, the second display area synchronously displays the relative position of the same target mapping point in the sagittal image. The computer device can synchronously display the target mapping point of the current puncture in real time in the sagittal image of the second display area by rotating the ultrasound probe until the target mapping point of the current puncture is located in the imaging area of ​​the sagittal plane in the cross-sectional image.

[0083] In the traditional prostate focal treatment process, after the computer equipment confirms the puncture position in the cross-sectional imaging, it rotates the ultrasound probe to cut to the puncture position with sagittal imaging. After the current puncture is completed, the ultrasound probe is rotated to the center position of the cross-sectional imaging, the next puncture position is confirmed and the above operation is repeated. Since the probe is rotated multiple times during the puncture process, it is impossible to simultaneously confirm whether the puncture position of the cross-sectional imaging is offset during the sagittal imaging monitoring and puncture process; and each puncture requires the ultrasound probe to be restored to the center position to ensure the mapping relationship between the puncture plate and the cross-sectional imaging, which is cumbersome. Therefore, how to display the same puncture position in both the cross-sectional imaging and the sagittal imaging at the same time, so as to accurately determine the spatial position relationship between the two planes, is a difficult problem to solve.

[0084] In contrast, in the embodiment of the present application, in response to the rotation operation of the ultrasound probe, during the rotation of the ultrasound probe, information such as the cross-sectional image, mapping points, and the imaging area of ​​the sagittal plane can be updated in real time. When any target mapping point coincides with the imaging area of ​​the sagittal plane, the relative position of the target mapping point located on the imaging area of ​​the sagittal plane in the sagittal plane image can be displayed in the second display area; thereby achieving that the same target mapping point can be displayed simultaneously in the cross-sectional image and the sagittal plane image. On the one hand, the puncture position of each puncture can be accurately determined, with relatively high accuracy and stability, reducing the occurrence of multiple punctures due to inaccurate puncture positions and improving patient experience; on the other hand, each puncture does not require repeated movement of the ultrasound probe to observe the puncture position. After the last puncture, the ultrasound probe can be directly rotated to the next target mapping point for puncture, simplifying the operation, saving time and improving puncture efficiency.

[0085] In some embodiments, the image processing method further comprises:

[0086] A first mark is added to the target mapping point located in the area to be ablated, and the marked target mapping point is displayed in the first display area and / or the second display area; the first mark includes at least one of a color mark, a graphic mark and a character string mark.

[0087] For example, the computer device may mark the target mapping point of the area to be ablated in green, and when the target mapping point is located in the imaging area of ​​the sagittal plane, the target mapping point marked in green may be simultaneously displayed in the cross-sectional image of the first display area and the sagittal image of the second display area. In this way, the target mapping point can be distinguished from other mapping points, and the target mapping point to be operated can be highlighted.

[0088] In the embodiment of the present application, since there are multiple mapping points of the puncture target, the target mapping points are marked, for example, the target mapping points are marked in yellow, and the mapping points other than the target mapping points are displayed in white; or the target mapping points are marked with graphics such as triangular frames, and the mapping points other than the target mapping points are displayed with circular frames. In this way, the target mapping points can be displayed more intuitively, which is convenient for medical staff to determine the position of the target mapping points from the mapping points and perform corresponding puncture operations according to the target mapping points.

[0089] In some embodiments, the image processing method further comprises:

[0090] In response to an operation request for a puncture operation, a target mapping point for the puncture is located; the target mapping point for the puncture is a target mapping point located on an imaging area of ​​a sagittal plane;

[0091] Add a second marker to the target mapping point of the current puncture.

[0092] In some embodiments, since there may be multiple target mapping points, the computer device can determine the target mapping point of the puncture at each time, and further identify the target mapping point of the puncture, for example, by identifying the target mapping point of the puncture with a blue circle.

[0093] In an embodiment of the present application, by further adding a second identifier to the target mapping point of the current puncture, the target mapping point of the current puncture can be displayed more intuitively, making it easier for medical staff to accurately locate the position of the target mapping point of the current puncture and complete the puncture operation based on the target mapping point of the current puncture.

[0094] In some embodiments, the image processing method further comprises:

[0095] When the ultrasonic probe and the puncture plate are orthogonal, the sensor coordinates, the first matrix and the second matrix of the positioning sensor in the corresponding space coordinate system are obtained through the positioning sensor connected to the ultrasonic probe; the first matrix is ​​a matrix determined according to the sensor coordinate transformation; the second matrix is ​​used to represent the first position transformation relationship between the positioning sensor and the ultrasonic probe;

[0096] determining a second position conversion relationship between the two-dimensional coordinate system and a spatial coordinate system corresponding to the positioning sensor according to the sensor coordinates, the first matrix and / or the second matrix;

[0097] When any one of the target mapping points is located in the imaging area of ​​the sagittal plane in the cross-sectional image, displaying in the second display area the relative position of the target mapping point located in the imaging area of ​​the sagittal plane in the sagittal plane image, comprises:

[0098] Determining coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the two-dimensional coordinate system according to the second position conversion relationship;

[0099] Determining target coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the three-dimensional coordinate system according to the coordinate data and the registration transformation matrix;

[0100] The relative position of the target mapping point located on the imaging area of ​​the sagittal plane in the sagittal plane image is displayed in the second display area according to the target coordinate data.

[0101] In some embodiments, the positioning sensor may be an electromagnetic tracker, and the spatial coordinate system may be a magnetic field coordinate system; the electromagnetic tracker senses the magnetic field and can calculate the coordinate information of the puncture plate and / or the guided ablation device in the magnetic field coordinate system. Alternatively, the positioning sensor may be an optical tracker, and the spatial coordinate system may be a three-dimensional coordinate system; by attaching a marker to the surgical instrument, the optical tracker illuminates the target object with infrared light, and then the infrared light is reflected from the marker back to the infrared sensor on the optical tracker, and the intersection of the light rays can be used to triangulate the three-dimensional coordinates of the marker of the target object.

[0102] For example, Figure 7 As shown, Figure 7 This is a schematic diagram showing the puncture plate and the ultrasound probe being orthogonal.

[0103] For example, when the ultrasound probe is rotated, Figure 8 As shown, Figure 8 This is a schematic diagram of the display after the ultrasound probe is rotated. At this time, the ultrasound probe and the puncture plate are in a non-orthogonal state.

[0104] Exemplarily, the sensor coordinates may be: ; A calculation method to determine the first matrix is: ;in, Indicates the rotational position matrix of the positioning sensor in the spatial coordinate system; the computer device can determine the second matrix in advance by measurement and / or calibration .

[0105] In the embodiment of the present application, the position conversion relationship may indicate the coordinate data conversion between two coordinate systems. For example, the second position conversion relationship may be a conversion from a two-dimensional coordinate system to a spatial coordinate system, or from a spatial coordinate system to a two-dimensional coordinate system.

[0106] Exemplarily, a method for determining target coordinate data of a target mapping point in a three-dimensional coordinate system is: in, indicates the registration transformation matrix; Indicates the coordinate data of the target mapping point in the two-dimensional coordinate system; Indicates the target coordinate data of the target mapping point in the three-dimensional coordinate system.

[0107] In some embodiments, the computer device can update the coordinate data of any target mapping point in the two-dimensional coordinate system in the updated cross-sectional image in real time according to the second position conversion relationship, so as to accurately determine the coordinate data of the target mapping point located in the imaging area of ​​the sagittal plane during the puncture (i.e., the target mapping point of the puncture) in the three-dimensional coordinate system.

[0108] In an embodiment of the present application, by determining the second position conversion relationship between the two-dimensional coordinate system and the spatial coordinate system corresponding to the positioning sensor connected to the ultrasound probe when the ultrasound probe is orthogonal to the puncture plate, the coordinate data of any target mapping point in the two-dimensional coordinate system in the updated cross-sectional image can be updated in real time during the rotation of the ultrasound probe according to the second position conversion relationship, thereby achieving precise positioning of the puncture position and improving the precision and accuracy of the puncture guidance.

[0109] In some embodiments, determining a second position conversion relationship between the two-dimensional coordinate system and the spatial coordinate system corresponding to the positioning sensor according to the sensor coordinates, the first matrix and / or the second matrix includes at least one of the following:

[0110] The first method: obtaining a first coordinate of the target mapping point in the two-dimensional coordinate system, and determining a second coordinate of the target mapping point in the spatial coordinate system according to the first matrix, the second matrix and the first coordinate;

[0111] The second method: obtaining the vertex coordinates of the vertices of the cross-sectional image in the spatial coordinate system according to the first matrix, the second matrix and the size of the cross-sectional image; and determining the second coordinates of the target mapping point in the spatial coordinate system according to the first coordinates and the vertex coordinates.

[0112] Exemplarily, a method of determining the second coordinate according to the first matrix, the second matrix and the first coordinate may be: ;in, Indicates the first matrix; Indicates the second matrix; Indicates the second coordinate of the target mapping point in the two-dimensional coordinate system in the cross-sectional image.

[0113] For example, Fig. 9 As shown, Fig. 9 A method for calculating the vertex coordinates of the vertices of the cross-sectional image in a spatial coordinate system includes:

[0114]

[0115] in, Indicates the first matrix; Indicates the second matrix; Indicates the length of the cross-sectional image; Indicates the width of the cross-sectional image; Indicates the coordinates of the upper left vertex in the cross-sectional image; Indicates the coordinates of the upper right vertex in the cross-sectional image; Indicates the coordinates of the lower left vertex in the cross-sectional image; Indicates the coordinates of the lower right vertex in the cross-sectional image.

[0116] Exemplarily, a method of determining the second coordinate according to the first coordinate and the vertex coordinate includes:

[0117]

[0118] in, Indicates the coordinates of the upper left vertex in the cross-sectional image; Indicates the coordinates of the upper right vertex in the cross-sectional image; Indicates the coordinates of the lower left vertex in the cross-sectional image; , Indicates the first coordinate of the target mapping point in the two-dimensional coordinate system in the cross-sectional image.

[0119] In some embodiments, the two methods of determining the second coordinate in the above embodiments determine the conversion relationship between the first coordinate in the two-dimensional coordinate system and the second coordinate in the spatial coordinate system, thereby establishing a second position conversion relationship; based on the second position conversion relationship, the spatial position conversion between the two coordinate systems can be achieved.

[0120] In an embodiment of the present application, a second position conversion relationship between the two-dimensional coordinate system corresponding to the ultrasound sequence image and the spatial coordinate system of the positioning sensor can be accurately established in a variety of ways, thereby eliminating the need to move the ultrasound probe to the center position after each rotation of the ultrasound probe, thereby simplifying operations and improving efficiency.

[0121] In some embodiments, determining the coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the two-dimensional coordinate system according to the second position conversion relationship includes:

[0122] Updating the second matrix of the positioning sensor in the spatial coordinate system to a third matrix; the third matrix is ​​used to represent a third position conversion relationship between the positioning sensor and the rotated ultrasound probe;

[0123] According to the second coordinates of the target mapping point located on the imaging area of ​​the sagittal plane and the third matrix, the coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the two-dimensional coordinate system corresponding to the updated cross-sectional image is determined using the second position transformation relationship.

[0124] Exemplarily, a method of updating a first coordinate of a target mapping point located in an imaging area of ​​a sagittal plane in a two-dimensional coordinate system includes:

[0125]

[0126] in, Indicates the third matrix; indicating a second coordinate of a target mapping point; Indicates the horizontal coordinate in the first coordinate; Indicates the vertical coordinate in the first coordinate; Indicates the first coordinate.

[0127] In an embodiment of the present application, after rotating the ultrasound probe, the computer device can update the first coordinate of the target mapping point on the updated cross-sectional image through a second position conversion relationship between the two-dimensional coordinate system corresponding to the ultrasound sequence image and the spatial coordinate system corresponding to the positioning sensor connected to the ultrasound probe, while keeping the second coordinate of the target mapping point unchanged, thereby achieving precise positioning of the target mapping point and improving the accuracy and precision of puncture guidance.

[0128] In the embodiments of the present application, the following provides specific examples in combination with any of the above embodiments:

[0129] Specific example 1: Fig.10 is an image processing method shown as an example; Fig.10 As shown, the image processing method is executed by a computer device, and the image processing method includes the following steps:

[0130] S201, obtaining a sequence of medical images of a target object.

[0131] In an optional embodiment, the target object may be a prostate; the computer device acquires a sequence of medical images including a lesion area of ​​the target object, an area to be ablated, and a contour of the target object before surgery.

[0132] S202, acquiring ultrasound sequence images of the target object in real time through an ultrasound probe.

[0133] S203, registering the ultrasound image sequence with the medical image sequence to obtain a registration transformation matrix and a registered fused three-dimensional image.

[0134] In an optional embodiment, the registration transformation matrix is ​​used to indicate the spatial conversion relationship between the two-dimensional coordinate system corresponding to the ultrasound sequence images and the three-dimensional coordinate system corresponding to the medical image sequence images.

[0135] S204, displaying the cross-sectional image in the registered and fused three-dimensional image, the mapping points of the puncture points on the puncture plate projected on the cross-sectional image, and the imaging area of ​​the sagittal plane in the first display area.

[0136] In an optional embodiment, when the ultrasound probe is orthogonal to the puncture plate, the puncture point on the puncture plate can be projected onto the cross-sectional image; the projected mapping point on the cross-sectional image can accurately guide the prostate puncture operation.

[0137] S205, when the ultrasonic probe and the puncture plate are orthogonal, determine a second position conversion relationship between the two-dimensional coordinate system and the space coordinate system corresponding to the positioning sensor.

[0138] In an optional embodiment, the sensor coordinates, the first matrix and the second matrix of the positioning sensor in the corresponding spatial coordinate system are obtained through a positioning sensor connected to the ultrasonic probe; the first matrix is ​​a matrix determined according to the sensor coordinate transformation; the second matrix is ​​used to characterize the first position transformation relationship between the positioning sensor and the ultrasonic probe; according to the sensor coordinates, the first matrix and / or the second matrix, the second position transformation relationship between the two-dimensional coordinate system and the spatial coordinate system corresponding to the positioning sensor is determined.

[0139] S206, displaying the cross-sectional image in the registered and fused three-dimensional image, the mapping points of the puncture points on the puncture plate projected on the cross-sectional image, and the imaging area of ​​the sagittal plane in the first display area.

[0140] In an optional embodiment, a connector is provided on the ultrasound probe and a positioning sensor is bound thereto; a cross-sectional image, a mapping point of the puncture target, and an imaging area of ​​the sagittal plane are displayed in the first display area.

[0141] S207 , in response to the rotation operation of the ultrasound probe, updating the cross-sectional image, the mapping points, and the imaging area of ​​the sagittal plane in the first display area in real time according to the rotation operation.

[0142] S208 , when any target mapping point is located in the imaging area of ​​the sagittal plane in the cross-sectional image, the relative position of the target mapping point located in the imaging area of ​​the sagittal plane in the sagittal plane image is displayed in the second display area.

[0143] In an optional embodiment, the target mapping point indicates the mapping point closest to the puncture target point; the puncture target point is located in the contour of the area to be ablated. When the target mapping point of the current puncture is located in the imaging area of ​​the sagittal plane, the rotation of the ultrasound probe can be stopped; the coordinate data of the target mapping point of the current mapping in the two-dimensional coordinate system is updated through the second position conversion relationship; the target coordinate data of the target mapping point of the current puncture in the three-dimensional coordinate system is determined according to the registration transformation matrix and the updated coordinate data of the target mapping point of the current mapping; according to the target coordinate data, the relative position of the target mapping point of the current mapping in the sagittal plane image is displayed in the second display area.

[0144] In the above-mentioned image processing method, in response to the rotation operation of the ultrasound probe, during the rotation process of the ultrasound probe, information such as the cross-sectional image, mapping points and the imaging area of ​​the sagittal plane can be updated in real time. When any target mapping point coincides with the imaging area of ​​the sagittal plane, the relative position of the target mapping point located on the imaging area of ​​the sagittal plane in the sagittal plane image can be displayed in the second display area; thereby, the same target mapping point can be displayed simultaneously in the cross-sectional image and the sagittal plane image. On the one hand, the puncture position of each puncture can be accurately determined, with relatively high accuracy and stability, reducing the occurrence of multiple punctures due to inaccurate puncture positions and improving patient experience; on the other hand, each puncture does not need to repeatedly move the ultrasound probe to observe the puncture position. After the last puncture, the ultrasound probe can be directly rotated to the next target mapping point for puncture, simplifying the operation, saving time and improving puncture efficiency.

[0145] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0146] Based on the same inventive concept, the embodiment of the present application also provides a puncture guidance system for implementing the above-mentioned image processing method. The implementation solution provided by the system is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more puncture guidance system embodiments provided below can refer to the limitations of the image processing method above, and will not be repeated here.

[0147] In some embodiments, Fig.11 As shown, Fig.11 It is a schematic diagram of the structural block diagram of a puncture guidance system; the present application provides a puncture guidance system, including: an ultrasonic probe 110 and a control unit 120;

[0148] The ultrasound probe 110 is in communication connection with the control unit 120 and is used to send ultrasound sequence images of the target object to the control unit 120 in real time;

[0149] The control unit 120 includes:

[0150] An acquisition module, used to acquire a sequence of medical image images and the ultrasound sequence images of the target object; wherein the sequence of medical image images includes a lesion area, an area to be ablated, and a contour of the target object;

[0151] A registration module, used for performing registration based on the ultrasound sequence images and the medical image sequence images to obtain a registration transformation matrix and a three-dimensional image after registration fusion; the registration transformation matrix is ​​used to indicate the spatial conversion relationship between the two-dimensional coordinate system corresponding to the ultrasound sequence images and the three-dimensional coordinate system corresponding to the medical image sequence images;

[0152] A first processing module, used for displaying in a first display area the cross-sectional image in the registered and fused three-dimensional image, the relative position of the mapping point of the puncture point on the puncture plate projected on the cross-sectional image and the imaging area of ​​the sagittal plane;

[0153] The first processing module is further configured to respond to a rotation operation of the ultrasound probe and update the cross-sectional image, the mapping points and the imaging area of ​​the sagittal plane in the first display area in real time according to the rotation operation;

[0154] The second processing module is used to display the relative position of the target mapping point located in the imaging area of ​​the sagittal plane in the sagittal plane image in the second display area when any target mapping point is located in the imaging area of ​​the sagittal plane in the cross-sectional image; the target mapping point indicates the mapping point closest to the puncture target point; the puncture target point is located in the outline of the area to be ablated.

[0155] In some embodiments, the communication connection may include a wired connection and a wireless connection. For example, the ultrasound probe puncture unit and the navigation unit are both wiredly connected to the computer device through a physical medium, and data transmission is performed through a physical line, which has higher stability and speed. Alternatively, the ultrasound probe, the puncture unit, and the navigation unit are all wirelessly connected to the computer device through wireless technology, and data transmission is not limited by physical lines, and the flexibility is relatively better.

[0156] In some embodiments, the system further comprises: a puncture unit and a navigation unit;

[0157] The puncture unit includes an interventional ablation device and a puncture plate; the puncture plate is detachably arranged on the ultrasound probe 110;

[0158] The navigation unit includes a positioning sensor; the positioning sensor is connected to the ultrasound probe 110 via a connector.

[0159] In the embodiment of the present application, the interventional ablation device is a device that transmits energy to the lesion of the target object through a catheter or other small device to destroy the lesion. The interventional ablation device includes but is not limited to at least one of a radiofrequency ablation device, a microwave ablation device, a cryoablation device, and a chemical ablation device. For example, the interventional ablation device can be an ablation needle.

[0160] In some embodiments, the puncture plate includes a color ribbon and a plurality of puncture points arranged at intervals; the color ribbon is located between two adjacent puncture points.

[0161] For example, Fig.12 As shown, Fig.12 The puncture board is a schematic diagram. A color band and multiple puncture target points set at intervals can be set on the puncture board. The color band is located between two adjacent puncture target points. The color band can be set to different colors. Medical staff can judge the position of the puncture target point by simply observing the color band, so as to find the puncture target point corresponding to the target mapping point more intuitively, conveniently and accurately.

[0162] In some embodiments, the system further comprises:

[0163] A first determination unit is used to obtain sensor coordinates, a first matrix and a second matrix of the positioning sensor in a corresponding space coordinate system through a positioning sensor connected to the ultrasonic probe 110 when the ultrasonic probe 110 is orthogonal to the puncture plate; the first matrix is ​​a matrix determined by the sensor coordinate transformation; and the second matrix is ​​used to represent a first position transformation relationship between the positioning sensor and the ultrasonic probe 110;

[0164] A second determining unit, configured to determine a second position conversion relationship between the two-dimensional coordinate system and a spatial coordinate system corresponding to the positioning sensor according to the sensor coordinates, the first matrix and / or the second matrix;

[0165] The second processing module comprises:

[0166] A first processing submodule, configured to determine coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the two-dimensional coordinate system according to the second position conversion relationship;

[0167] A second processing submodule is used to determine the target coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the three-dimensional coordinate system according to the coordinate data and the registration transformation matrix;

[0168] The display submodule is used to display the relative position of the target mapping point located on the imaging area of ​​the sagittal plane in the sagittal plane image in the second display area according to the target coordinate data.

[0169] In some embodiments, the second determining unit is configured to perform at least one of the following steps:

[0170] The first method: obtaining the first coordinate of the target mapping point in the two-dimensional coordinate system, and determining the second coordinate of the target mapping point in the spatial coordinate system according to the first matrix, the second matrix and the first coordinate; establishing the second position conversion relationship according to the first coordinate and the second coordinate;

[0171] The second method: obtaining the vertex coordinates of the vertices of the cross-sectional image in the spatial coordinate system according to the first matrix, the second matrix and the size of the cross-sectional image; determining the second coordinates of the target mapping point in the spatial coordinate system according to the first coordinates and the vertex coordinates; establishing the second position conversion relationship according to the first coordinates and the second coordinates.

[0172] In some embodiments, the first processing submodule is used to perform the following steps:

[0173] Updating the second matrix of the positioning sensor in the spatial coordinate system to a third matrix; the third matrix is ​​used to represent a third position conversion relationship between the positioning sensor and the rotated ultrasound probe 110;

[0174] According to the second coordinates of the target mapping point located on the imaging area of ​​the sagittal plane and the third matrix, the coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the two-dimensional coordinate system corresponding to the updated cross-sectional image is determined using the first position transformation relationship.

[0175] Each unit or module in the above-mentioned puncture guidance system can be implemented in whole or in part by software, hardware or a combination thereof. Each unit or module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the corresponding operations of each unit or module.

[0176] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.13 As shown. The computer device includes a processor, a memory, a communication interface, a display unit and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an image processing method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse, etc.

[0177] Those skilled in the art will understand that Fig.13 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0178] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented: When the computer program executes the computer program, the steps in the above method embodiments are implemented.

[0179] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0180] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0181] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0182] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0183] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the following steps are implemented: Acquire a sequence of medical image images of the target object; wherein the sequence of medical image images includes a lesion area, an area to be ablated, and a contour of the target object; Acquiring a sequence of ultrasonic images of the target object in real time through an ultrasonic probe; Registering the ultrasound sequence images with the medical image sequence images to obtain a registration transformation matrix and a three-dimensional image after registration fusion; the registration transformation matrix is ​​used to indicate the spatial conversion relationship between the two-dimensional coordinate system corresponding to the ultrasound sequence images and the three-dimensional coordinate system corresponding to the medical image sequence images; Displaying in the first display area the cross-sectional image in the registered and fused three-dimensional image, the mapping point where the puncture point on the puncture plate is projected on the cross-sectional image, and the imaging area of ​​the sagittal plane; In response to the rotation operation of the ultrasound probe, updating the cross-sectional image, the mapping points, and the imaging area of ​​the sagittal plane in the first display area in real time according to the rotation operation; When any target mapping point is located in the imaging area of ​​the sagittal plane in the cross-sectional image, the relative position of the target mapping point located in the imaging area of ​​the sagittal plane in the sagittal image is displayed in the second display area; the target mapping point indicates the mapping point closest to the puncture target point; the puncture target point is located in the contour of the area to be ablated.

2. The computer device according to claim 1, characterized in that When the processor executes the computer program, the processor further implements the following steps: When the ultrasonic probe and the puncture plate are orthogonal, the sensor coordinates, the first matrix and the second matrix of the positioning sensor in the corresponding space coordinate system are obtained through the positioning sensor connected to the ultrasonic probe; the first matrix is ​​a matrix determined according to the sensor coordinate transformation; The second matrix is ​​used to represent a first position conversion relationship between the positioning sensor and the ultrasound probe; determining a second position conversion relationship between the two-dimensional coordinate system and a spatial coordinate system corresponding to the positioning sensor according to the sensor coordinates, the first matrix and / or the second matrix; When any one of the target mapping points is located in the imaging area of ​​the sagittal plane in the cross-sectional image, displaying the relative position of the target mapping point located in the imaging area of ​​the sagittal plane in the sagittal plane image in the second display area includes: Determining coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the two-dimensional coordinate system according to the second position conversion relationship; Determining target coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the three-dimensional coordinate system according to the coordinate data and the registration transformation matrix; The relative position of the target mapping point located on the imaging area of ​​the sagittal plane in the sagittal plane image is displayed in the second display area according to the target coordinate data.

3. The computer device according to claim 2, characterized in that Determining a second position conversion relationship between the two-dimensional coordinate system and the spatial coordinate system corresponding to the positioning sensor according to the sensor coordinates, the first matrix and / or the second matrix includes at least one of the following: The first method: obtaining the first coordinate of the target mapping point in the two-dimensional coordinate system, and determining the second coordinate of the target mapping point in the spatial coordinate system according to the first matrix, the second matrix and the first coordinate; establishing the second position conversion relationship according to the first coordinate and the second coordinate; The second method: obtaining the vertex coordinates of the vertices of the cross-sectional image in the spatial coordinate system according to the first matrix, the second matrix and the size of the cross-sectional image; determining the second coordinates of the target mapping point in the spatial coordinate system according to the first coordinates and the vertex coordinates; establishing the second position conversion relationship according to the first coordinates and the second coordinates.

4. The computer device according to claim 3, characterized in that Determining the coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the two-dimensional coordinate system according to the second position conversion relationship includes: Updating the second matrix of the positioning sensor in the spatial coordinate system to a third matrix; the third matrix is ​​used to represent a third position conversion relationship between the positioning sensor and the rotated ultrasound probe; According to the second coordinates of the target mapping point located on the imaging area of ​​the sagittal plane and the third matrix, the coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the two-dimensional coordinate system corresponding to the updated cross-sectional image is determined using the second position transformation relationship.

5. A puncture guide system, characterized in that: include: Ultrasound probe and control unit; The ultrasound probe is communicatively connected with the control unit and is used to send ultrasound sequence images of the target object to the control unit in real time; The control unit comprises: An acquisition module, used to acquire a sequence of medical image images and the ultrasound sequence images of the target object; wherein the sequence of medical image images includes a lesion area, an area to be ablated, and a contour of the target object; A registration module, used for performing registration based on the ultrasound sequence images and the medical image sequence images to obtain a registration transformation matrix and a three-dimensional image after registration fusion; the registration transformation matrix is ​​used to indicate the spatial conversion relationship between the two-dimensional coordinate system corresponding to the ultrasound sequence images and the three-dimensional coordinate system corresponding to the medical image sequence images; A first processing module, used for displaying in a first display area the cross-sectional image in the registered and fused three-dimensional image, the relative position of the mapping point of the puncture point on the puncture plate projected on the cross-sectional image and the imaging area of ​​the sagittal plane; The first processing module is further configured to respond to a rotation operation of the ultrasound probe and update the cross-sectional image, the mapping points and the imaging area of ​​the sagittal plane in the first display area in real time according to the rotation operation; The second processing module is used to display the relative position of the target mapping point located in the imaging area of ​​the sagittal plane in the sagittal image in the second display area when any target mapping point is located in the imaging area of ​​the sagittal plane in the cross-sectional image; the target mapping point indicates the mapping point closest to the puncture target point; the puncture target point is located in the outline of the area to be ablated.

6. The puncture guide system according to claim 5, characterized in that: The system further comprises: a puncture unit and a navigation unit; The puncture unit includes an interventional ablation device and a puncture plate; the puncture plate is detachably arranged on the ultrasound probe; The navigation unit includes a positioning sensor; the positioning sensor is connected to the ultrasound probe via a connector.

7. The puncture guide system according to claim 6, characterized in that: The puncture plate comprises a color ribbon and a plurality of puncture points arranged at intervals; the color ribbon is located between two adjacent puncture points.

8. The puncture guide system according to claim 5, characterized in that: The system further comprises: A first determination unit is used to obtain sensor coordinates, a first matrix and a second matrix of the positioning sensor in a corresponding space coordinate system through a positioning sensor connected to the ultrasound probe when the ultrasound probe and the puncture plate are orthogonal; the first matrix is ​​a matrix determined according to the sensor coordinate transformation; the second matrix is ​​used to represent a first position transformation relationship between the positioning sensor and the ultrasound probe; a second determining unit, configured to determine a second position conversion relationship between the two-dimensional coordinate system and a spatial coordinate system corresponding to the positioning sensor according to the sensor coordinates, the first matrix and / or the second matrix; The second processing module comprises: A first processing submodule, configured to determine coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the two-dimensional coordinate system according to the second position conversion relationship; A second processing submodule is used to determine the target coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the three-dimensional coordinate system according to the coordinate data and the registration transformation matrix; The display submodule is used to display the relative position of the target mapping point located on the imaging area of ​​the sagittal plane in the sagittal plane image in the second display area according to the target coordinate data.

9. The puncture guide system according to claim 8, characterized in that: The second determining unit is configured to perform at least one of the following steps: The first method: obtaining the first coordinate of the target mapping point in the two-dimensional coordinate system, and determining the second coordinate of the target mapping point in the spatial coordinate system according to the first matrix, the second matrix and the first coordinate; establishing the second position conversion relationship according to the first coordinate and the second coordinate; The second method: obtaining the vertex coordinates of the vertices of the cross-sectional image in the spatial coordinate system according to the first matrix, the second matrix and the size of the cross-sectional image; determining the second coordinates of the target mapping point in the spatial coordinate system according to the first coordinates and the vertex coordinates; establishing the second position conversion relationship according to the first coordinates and the second coordinates.

10. The puncture guide system according to claim 9, characterized in that: The first processing submodule is used to perform the following steps: Updating the second matrix of the positioning sensor in the spatial coordinate system to a third matrix; the third matrix is ​​used to represent a third position conversion relationship between the positioning sensor and the rotated ultrasound probe; According to the second coordinates of the target mapping point located on the imaging area of ​​the sagittal plane and the third matrix, the coordinate data of the target mapping point located on the imaging area of ​​the sagittal plane in the two-dimensional coordinate system corresponding to the updated cross-sectional image is determined using the first position transformation relationship.

Images