Image registration method and related devices based on ultrasonic probe

Abstract

This application provides an image registration method and related apparatus based on an ultrasound probe. When the ultrasound probe scans a target organ, an electronic device receives a first command sent by the ultrasound probe, triggered by a first user operation on a button assembly. Starting from the moment the first command is received, the electronic device reconstructs multiple target ultrasound images acquired by the ultrasound device until a second command is received from the ultrasound probe, obtaining the ultrasound contour of the target organ. The second command is triggered by a second user operation on the button assembly, and the acquisition time of the target ultrasound image is between the triggering times of the first and second commands. The electronic device performs registration based on the ultrasound contour and CT contour of the target organ to obtain a registration result. The CT contour of the target organ is segmented from the acquired CT images. This application simplifies the image registration process.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to an image registration method and related apparatus based on an ultrasonic probe. Background Technology

[0002] With the continuous development of computer science and technology, image registration technology has become increasingly mature. For example, in kidney-related puncture surgery, it is necessary to register the ultrasound contour of the kidney obtained from ultrasound images with the CT contour of the kidney obtained from computed tomography (CT) images, so that surgeons and other operators can obtain information about the inside of the kidney during the operation, thereby reducing surgical errors.

[0003] During the process of scanning a target organ with an ultrasound probe, the acquired ultrasound images may not all be suitable for registration with CT images. Therefore, the operator needs to use one hand to move the ultrasound probe to scan the target organ and the other hand to use a mouse or other device to select the ultrasound image suitable for registration with the CT image, which is inconvenient. Summary of the Invention

[0004] This application provides an image registration method and related apparatus based on an ultrasonic probe, which can simplify the image registration process.

[0005] In this embodiment, the ultrasound probe includes a button assembly. During ultrasound image scanning, when the ultrasound probe detects a target organ, the user can directly perform a first user operation on the button assembly to send a first command to the electronic device. This command causes the electronic device to use the target ultrasound image acquired between the triggering time of the first command and the triggering time of the second command to reconstruct the contour of the target organ. This process continues until the electronic device receives a second command from the ultrasound probe, thus obtaining the ultrasound contour of the target organ. Then, registration is performed based on the ultrasound contour and the CT contour of the target organ to obtain the registration result. Throughout this reconstruction and registration process, the target ultrasound image used for reconstruction is controlled by the button assembly on the ultrasound probe. This means the user only needs to use the button assembly on the ultrasound probe to select a suitable target ultrasound image for ultrasound contour reconstruction, without needing to use the control device of the electronic device (such as an input device like a mouse). This makes the operation simple, convenient, and efficient.

[0006] In a first aspect, embodiments of this application provide an image registration method based on an ultrasound probe. This method is applied to an electronic device in a surgical navigation system, which includes the electronic device, an optical tracking device, and an ultrasound device. The ultrasound probe of the ultrasound device includes a button assembly, and at least N markers are disposed on the ultrasound probe. These N markers are within the tracking range of the optical tracking device, which positions the ultrasound probe by tracking the N markers, where N is an integer greater than or equal to 3. The method includes:

[0007] When the ultrasound probe scans the target organ, a first instruction sent by the ultrasound probe is received. The first instruction is used to indicate the start of image acquisition and reconstruction. The first instruction is triggered by a first user operation on the button assembly.

[0008] Starting from the moment the first instruction is received, multiple target ultrasound images acquired by the ultrasound device are reconstructed until the second instruction sent by the ultrasound probe is received, and the ultrasound contour of the target organ is obtained; the second instruction is triggered by a second user operation on the button assembly, and the second instruction is used to indicate the end of image acquisition and reconstruction; the acquisition time of the target ultrasound image is between the triggering time of the first instruction and the triggering time of the second instruction.

[0009] Based on the ultrasound profile and the computed tomography (CT) profile of the target organ, registration is performed to obtain the registration result. The CT profile of the target organ is segmented from the obtained CT images.

[0010] In conjunction with the first aspect, in some embodiments, before receiving the first instruction sent by the ultrasound probe, the method further includes:

[0011] Receive the third instruction sent by the ultrasound probe, the third instruction being used to instruct position calibration, the third instruction being triggered by a third user operation on the button assembly.

[0012] A first reference direction is determined by transforming the first direction vector from the CT coordinate system to the world coordinate system; a second reference direction is determined by transforming the second direction vector from the CT coordinate system to the world coordinate system; and a third reference direction is determined by transforming the third direction vector from the CT coordinate system to the world coordinate system. The first, second, and third reference directions are common to the space corresponding to the ultrasound equipment and the space corresponding to the CT equipment. The CT coordinate system is the coordinate system corresponding to the CT equipment.

[0013] Based on the spatial position information of the first ultrasound image within the tracking range of the aforementioned optical tracking device, the fourth direction vector of the first reference direction, which is transformed from the ultrasound coordinate system to the world coordinate system, the fifth direction vector of the second reference direction, which is transformed from the ultrasound coordinate system to the world coordinate system, and the sixth direction vector of the third reference direction, which is transformed from the ultrasound coordinate system to the world coordinate system, are determined; the ultrasound coordinate system is the coordinate system corresponding to the ultrasound device, and the acquisition time of the first ultrasound image is the trigger time of the third command.

[0014] Based on the first angle and first position offset between the first direction vector and the fourth direction vector, the second angle and second position offset between the second direction vector and the fifth direction vector, and the third angle and third position offset between the third direction vector and the sixth direction vector, the rotation and translation matrix is ​​determined.

[0015] The aforementioned rotation and translation matrix is ​​used to transform the initial CT image, resulting in the CT image obtained above.

[0016] In conjunction with the first aspect, in some embodiments, the above method further includes:

[0017] During the needle insertion point determination stage, when the target point is scanned using the aforementioned ultrasonic probe, a fourth instruction sent by the aforementioned ultrasonic probe is received. The fourth instruction is used to indicate the selection of the needle insertion point. The fourth instruction is triggered by a fourth user operation on the aforementioned button assembly. The needle insertion point determination stage is later than the aforementioned image registration stage.

[0018] The coordinates of the target point on the ultrasound probe at the first moment are used as the coordinates of the needle insertion point. The target point is located on the line segment where the scanning surface of the ultrasound probe intersects with the ultrasound probe. The first moment is the triggering moment of the fourth command.

[0019] In conjunction with the first aspect, in some embodiments, after using the coordinates of the target point on the ultrasound probe at a first moment as the coordinates of the needle insertion point, the method further includes:

[0020] The first puncture path is determined based on the aforementioned needle insertion point and the aforementioned target point; the first puncture path includes the aforementioned needle insertion point and the aforementioned target point.

[0021] Determine the first plane corresponding to the currently acquired second ultrasound image;

[0022] In the case where the first puncture path is included on the first plane, the electronic device transforms the first puncture path based on a transformation matrix to obtain a second puncture path; the transformation matrix is ​​determined based on the spatial positions of N markers on the ultrasound probe.

[0023] The second puncture path is superimposed on the second ultrasound image.

[0024] In conjunction with the first aspect, in some embodiments, the second puncture path includes a modified needle insertion point and a modified target point; the superimposition of the second puncture path on the target ultrasound image includes:

[0025] The second puncture path, the transformed needle insertion point, and the transformed target point are superimposed on the second ultrasound image.

[0026] Secondly, embodiments of this application provide an image registration device based on an ultrasound probe, including a unit for performing the method in the first aspect or any possible implementation thereof.

[0027] Thirdly, embodiments of this application provide a surgical navigation system, which includes an electronic device, an optical tracking device, and an ultrasound device; the ultrasound probe of the ultrasound device includes a button assembly, and at least N markers are provided on the ultrasound probe, the N markers being within the tracking range of the optical tracking device, and the optical tracking device locating the ultrasound probe by tracking the N markers; N is an integer greater than or equal to 3;

[0028] The aforementioned electronic device is used to receive a first instruction sent by the ultrasound probe when the ultrasound probe scans a target organ. The first instruction is used to indicate the start of image acquisition and reconstruction. The first instruction is triggered by a first user operation on the button assembly.

[0029] The aforementioned electronic device is used to reconstruct multiple target ultrasound images acquired by the aforementioned ultrasound device starting from the moment the first instruction is received, until the second instruction sent by the aforementioned ultrasound probe is received, thereby obtaining the ultrasound contour of the target organ; the aforementioned second instruction is triggered by a second user operation on the aforementioned button assembly, and the aforementioned second instruction is used to indicate the end of image acquisition and reconstruction, and the acquisition time of the aforementioned target ultrasound images is between the triggering time of the aforementioned first instruction and the triggering time of the aforementioned second instruction.

[0030] The aforementioned electronic device is used to register the ultrasound profile of the target organ and the computed tomography (CT) profile of the target organ to obtain a registration result, wherein the CT profile of the target organ is segmented from the obtained CT image.

[0031] Fourthly, embodiments of this application disclose an electronic device, including: a processor and a memory, wherein the memory stores a computer program, and the processor calls the computer program stored in the memory to perform a method as described in the first aspect or any possible implementation of the first aspect.

[0032] Fifthly, this application also provides another electronic device, comprising: a processor, a transmitting device, an input device, an output device, and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs a method as described in the first aspect or any possible implementation thereof.

[0033] In a sixth aspect, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on one or more processors, causes the method as described in the first aspect or any possible implementation thereof to be executed.

[0034] In a seventh aspect, embodiments of this application provide a computer program product including program instructions that, when executed by a processor, cause the processor to perform a method as described in the first aspect or any possible implementation thereof. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of a surgical navigation system provided in an embodiment of this application;

[0036] Figure 2 This is a schematic diagram of an ultrasonic probe including a button assembly provided in an embodiment of this application;

[0037] Figure 3 This is a schematic flowchart of an image registration method based on an ultrasonic probe provided in an embodiment of this application;

[0038] Figure 4 This is a schematic diagram of an ultrasonic scan using an ultrasonic probe provided in an embodiment of this application;

[0039] Figure 5a This is a schematic diagram of a first puncture path on a first plane provided in an embodiment of this application;

[0040] Figure 5b This is a schematic diagram showing the puncture path provided in an embodiment of this application;

[0041] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0042] Figure 7 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this application. Detailed Implementation

[0043] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items. The terms “first” and “second,” etc., in the specification, claims, and drawings of this application are used to distinguish different objects and not to describe a particular order.

[0044] It should be noted that the terms "first," "second," "third," and "fourth," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. It should also be understood that the numbering before the steps in the embodiments of this application is for ease of understanding and description of the scheme, and should not be construed as a limitation on the order in which the steps are performed.

[0045] In this embodiment, the target organ can be understood as an organ including the target point, which can be scanned by ultrasound to obtain an ultrasound image or by CT scan to obtain a CT image. The target point can be understood as the location of the area to be punctured, such as the location of a lesion, such as a stone; or it can be a target location planned during the experiment. In this embodiment, the target point is determined from the CT image, and the target point that actually exists in the body of the subject can be called the actual target point.

[0046] It's understandable that CT images, with their high resolution, provide more detailed information about the internal structure of the target organ. Ultrasound images offer advantages such as real-time processing and low cost, but their resolution is lower than CT images. Therefore, a CT scan is generally required before surgery to obtain CT images of the target organ. Because of the high resolution of CT images, the target point is selected from the CT image; therefore, the target point lies in the CT coordinate system.

[0047] During surgery, due to the real-time nature of ultrasound images, it is necessary to convert the ultrasound images of the target organ to the world coordinate system for registration. This allows the target points, which were originally located in the CT coordinate system, to be displayed together in the accurate position on the ultrasound image, so that the operator can intuitively observe the target points in real time.

[0048] In this embodiment, the CT coordinate system can be understood as the coordinate system corresponding to the CT device, that is, the coordinate system used by the CT device. The ultrasound coordinate system can be understood as the coordinate system corresponding to the ultrasound device, that is, the coordinate system used by the ultrasound device.

[0049] Understandably, when using an ultrasound probe to scan a subject and acquire ultrasound images, the initial ultrasound images may not be suitable for registration, for example, the target organ may not be detected initially. In one approach, when selecting an ultrasound image for registration, the operator needs to move the ultrasound probe across the subject's body surface to acquire ultrasound images, observe the ultrasound images displayed on the monitor, and use an input device (such as a mouse) to select the ultrasound image for registration. This process is complex and inconvenient.

[0050] To address the aforementioned issues, this application provides an image registration method and related apparatus based on an ultrasound probe. This method simplifies the image registration process, making it more convenient and efficient. The method provided in this application can be executed by an electronic device in a surgical navigation system. This electronic device can be any type of electronic device capable of executing the technical solutions disclosed in the method embodiments of this application. The electronic device can be a tablet computer, PDA, laptop computer, etc., or it can be a terminal device, a server, or a server cluster composed of multiple servers, etc., and is not limited thereto. Optionally, the method embodiments of this application can also be implemented by a processor executing computer program code.

[0051] For example, please refer to Figure 1 , Figure 1 This is a schematic diagram of a surgical navigation system provided in an embodiment of this application.

[0052] like Figure 1 As shown, the surgical navigation system includes an electronic device 101, an optical tracking device 102, and an ultrasound device 103.

[0053] In this embodiment, the optical tracking device 102 can be understood as a device for measuring the spatial position information (such as spatial coordinates) of a marker. Exemplarily, the optical tracking device 102 may include a first sensor 1021 and a second sensor 1022. The first sensor 1021 may include a first infrared light-emitting diode and a first infrared receiver, and the second sensor 1022 may include a second infrared light-emitting diode and a second infrared receiver. When locating a marker within the tracking range, the first and second infrared light-emitting diodes generate infrared light, which is then irradiated onto the marker. The reflective coating on the marker's surface reflects the infrared light back to the first and second infrared receivers, allowing the optical tracking device 102 to locate the marker using the infrared light.

[0054] In this embodiment, the ultrasound device 103 can be understood as a device that uses an ultrasonic beam to scan the object 104 to be scanned, then receives the reflected signal of the ultrasonic beam, processes the reflected signal to obtain an ultrasound image of the target organ within the object to be scanned. Exemplarily, the ultrasound device 103 may include an ultrasound probe 1031, a first communication unit, a first processor, and a first power supply. The first power supply provides power to the ultrasound device 103, the first communication unit is used for communication between the ultrasound device 103 and the electronic device 101, and the first processor is used to process the data acquired by the ultrasound probe 1031 to obtain an ultrasound image. In this embodiment, the ultrasound probe is provided with N markers, where N is an integer greater than or equal to 3.

[0055] In this embodiment, the ultrasound probe 1031 includes a button assembly, which includes a second communication unit, a second processing unit, and a second power supply. The second power supply powers the button assembly of the ultrasound probe 1031, the second communication unit communicates between the ultrasound probe 1031 and the electronic device 101, and the second processor processes data from the ultrasound probe to send instructions to the electronic device 101, such as subsequent first, second, third, and fourth instructions.

[0056] For ease of understanding, please refer to the example provided. Figure 2 , Figure 2 This is a schematic diagram of an ultrasonic probe including a button assembly provided in an embodiment of this application.

[0057] like Figure 2 The ultrasonic probe 200 shown includes a button 201, and markers are set on the ultrasonic probe 200 via an optical positioning bracket (not shown in the figure), for example... Figure 2 The four markers shown are marker 202, marker 203, marker 204, and marker 205.

[0058] Understandable, Figure 1 The system shown is merely an example. Figure 1 The positional relationships between the devices shown do not constitute any limitation. For example, the module in electronic device 101 used to implement the method provided in the embodiments of this application may be located in ultrasonic device 103, or the data processing function of ultrasonic device 103 may be coupled in electronic device 101.

[0059] To facilitate understanding of the methods provided in the embodiments of this application, the surgical procedure is first introduced.

[0060] (1) CT image acquisition stage: CT images of the target organ are acquired before surgery.

[0061] First, determine whether enhanced CT is needed based on the patient's condition. Have the patient lie on the CT scanning bed, then push the CT scanning window in to perform a CT scan to obtain the patient's CT image data, such as plain CT data and enhanced CT data.

[0062] (2) Target determination stage.

[0063] Target point identification based on CT images. For example, artificial intelligence algorithms can be used to segment and reconstruct the three-dimensional contour of the target organ from the acquired CT images; then, the target point can be identified from this three-dimensional contour. The selection of the target point can be determined by experienced doctors or other operators.

[0064] (3) Ultrasound image acquisition stage: Ultrasound images of the target organ are acquired during the operation, and the ultrasound contour of the target organ is reconstructed.

[0065] (4) Image registration stage: The CT contour of the target organ is registered with the ultrasound contour of the target organ. It should be understood that the registration between contours is essentially image registration.

[0066] (5) Needle entry point determination stage: Determine the coordinates of the needle entry point.

[0067] (6) Stage of determining the puncture path.

[0068] (7) Puncture stage.

[0069] For example, the puncture stage can be performed manually by the operator or with the help of auxiliary equipment such as a robotic arm.

[0070] Understandably, data processing differs at different stages during surgery, and switching between stages can be controlled by the user on an electronic device. For example, each stage can correspond to a different button, and the user can click different buttons to switch to the corresponding stage.

[0071] The method in this application involves communication between an electronic device and an ultrasonic probe. For ease of understanding, the method provided in this application will be described below with the electronic device and the ultrasonic probe as the execution subjects.

[0072] For example, please refer to Figure 3 , Figure 3 This is a schematic flowchart of an image registration method based on an ultrasonic probe provided in an embodiment of this application. Figure 3 The method shown is applied in the ultrasound image acquisition and image registration stages described above, including:

[0073] 301: When the ultrasound probe scans the target organ, in response to a first user operation on the button assembly, the ultrasound probe sends a first command to the electronic device, the first command indicating the start of image acquisition and reconstruction. Accordingly, the electronic device receives the first command.

[0074] Understandably, an ultrasound probe scans the target object, acquiring and displaying ultrasound images in real time. Due to the complexity of the target object's internal structure, the operator can determine whether the target organ has been scanned by observing the ultrasound image on the monitor. Generally, this may require adjustments such as moving the ultrasound probe multiple times and changing the pressure applied to the probe to ensure the target organ is scanned. Understandably, although the operator can acquire ultrasound images in real time by moving the ultrasound probe, the electronic device does not use all acquired ultrasound images for reconstruction. Instead, it selects the ultrasound images that scan the target organ and have good image quality for reconstruction and subsequent registration.

[0075] In this step, when the ultrasound probe scans the target organ, the operator performs a first user operation on the button assembly, such as a pressing operation. In response to the first user operation, the ultrasound probe sends a first instruction to the electronic device, which instructs the start of image acquisition and reconstruction. The specific content of this first instruction can be set according to actual conditions, as long as the electronic device can be triggered to start ultrasound image acquisition and reconstruction upon receiving the first instruction. In this embodiment, the ultrasound probe sending a first instruction to the electronic device in response to the first user operation can be understood as the first instruction being triggered by the first user operation performed on the button assembly.

[0076] 302: The electronic device begins to reconstruct multiple target ultrasound images acquired by the ultrasound device at the moment it receives the first instruction, until it receives the second instruction sent by the ultrasound probe, and obtains the ultrasound contour of the target organ; the second instruction is triggered by a second user operation on the button assembly, and the second instruction is used to indicate the end of image acquisition and reconstruction, and the acquisition time of the target ultrasound image is between the triggering time of the first instruction and the triggering time of the second instruction.

[0077] It is understandable that the target organ is a three-dimensional structure, requiring multiple ultrasound scans from different locations to comprehensively scan the entire organ. Therefore, ultrasound image-based reconstruction can be understood as a process of duration T, during which the ultrasound probe needs to continuously move across the surface of the target. In this embodiment, the electronic device begins image acquisition and reconstruction upon receiving the first instruction and continues until it receives the second instruction from the ultrasound probe. Therefore, the second instruction can be understood as a termination instruction to stop ultrasound image acquisition and reconstruction.

[0078] In this step, the acquisition time of the target ultrasound image is between the triggering time of the first command and the triggering time of the second command. The reconstruction of the multiple target ultrasound images acquired by the electronic device can be understood as real-time reconstruction. That is, after each target ultrasound image is acquired, contour segmentation processing is performed. After the above segmentation processing of multiple ultrasound images, the complete ultrasound contour of the target organ can be obtained.

[0079] For example, the operator can move the ultrasound probe to acquire ultrasound images while simultaneously observing the real-time reconstruction results on the display. If the reconstructed kidney contour shows a half-kidney shape, the ultrasound contour reconstruction of the target organ can be considered complete, meaning the ultrasound contour of the target organ has been obtained. The user then performs the aforementioned second user operation on the button assembly of the ultrasound probe; this second user operation could be, for example, a pressing operation. For example, the reconstruction results can be displayed in real-time on the display. When the displayed reconstruction result shows a half-kidney shape, a pressing operation (i.e., the second user operation) is performed on the button assembly of the ultrasound probe to end the ultrasound reconstruction.

[0080] 303: The electronic device performs registration based on the ultrasound profile and CT profile of the target organ to obtain the registration result. The CT profile of the target organ is segmented from the obtained CT image.

[0081] It is understandable that after step 302, the electronic device has obtained the ultrasound profile of the target organ. By registering the ultrasound profile of the target organ with the CT profile of the target organ, the registration result can be obtained.

[0082] Operators can evaluate the registration results in the following ways:

[0083] Understandably, the main goal of image registration is to ensure that the CT and ultrasound contours of the target organ are precisely aligned, allowing the target points determined based on the CT image to be displayed together in their accurate locations on the ultrasound image (such as the location of the lesion), enabling the operator to visually observe the target points in real time. Here, the CT contour can be understood as the contour segmented from the CT image of the target organ, and the ultrasound contour can be understood as the contour segmented from the ultrasound image of the target organ.

[0084] On the one hand, the cross-union ratio (CUI) of the CT and ultrasound contours can be used to initially determine whether registration is complete. For example, if the CUI is greater than or equal to a threshold, registration is considered initially complete. On the other hand, lesions within the patient's body generally exhibit certain characteristics after ultrasound scanning; for example, stones appear brighter (higher grayscale value) in ultrasound images. However, due to the complexity of the patient's condition, other tissues may exhibit similar characteristics to the lesion in the ultrasound image. Therefore, the operator needs to judge based on the ultrasound image displayed on the screen and the real-time registered target point. If the real-time registered target point overlaps with the actual lesion (or the overlap is greater than a certain threshold), then registration is considered complete, and the next stage can proceed. It is understandable that if the target point in the registration result differs too much from the actual lesion, the current registration result can be discarded, and ultrasound acquisition, reconstruction, and registration can be performed again.

[0085] In this step, the acquired CT image can be understood as a CT image already existing in the electronic device. It should be understood that the acquired CT image was obtained by scanning the target organ, and therefore, the acquired CT image includes the target organ. In some embodiments, the acquired CT image may be the initial CT image obtained in the preceding CT image acquisition stage. In other embodiments, the acquired CT image may be an image obtained by transforming the initial image.

[0086] In some embodiments, Figure 3 In the method shown, before step 301, where the electronic device receives the first instruction sent by the ultrasound probe, the method further includes:

[0087] 304: The electronic device receives a third instruction from the ultrasonic probe, which is used to instruct position calibration. The third instruction is triggered by a third user operation on the button assembly.

[0088] This embodiment can be understood as a position calibration stage before image registration, or it can be called a coarse registration stage. For example, the electronic device can determine that it is currently in the position calibration stage via a button corresponding to the position calibration stage. During the position calibration stage, user operations on the button assembly trigger the ultrasound probe to send a third command.

[0089] 305: The electronic device determines a first reference direction transformed from the CT coordinate system to the world coordinate system as a first direction vector, a second reference direction transformed from the CT coordinate system to the world coordinate system as a second direction vector, and a third reference direction transformed from the CT coordinate system to the world coordinate system as a third direction vector; the first reference direction, the second reference direction, and the third reference direction are shared by the space corresponding to the ultrasound device and the space corresponding to the CT device.

[0090] In this embodiment, the first reference direction, the second reference direction, and the third reference direction can be collectively referred to as reference directions. The space corresponding to the ultrasound device and the space corresponding to the CT device are shared by both reference directions. This means that the reference direction can be determined as a direction vector in the space corresponding to the ultrasound device, or in the space corresponding to the CT device.

[0091] It is understandable that both ultrasound and CT images are acquired based on the subject. Even if the time and spatial information of ultrasound and CT image acquisition differ, the relative relationship between the target organ and the subject is fixed. For example, when acquiring a CT image, the relative position between the CT image and the subject is fixed; similarly, when acquiring an ultrasound image, the relative position between the ultrasound image and the subject is fixed. Therefore, the reference direction can be determined based on the subject, such as the direction from the subject's feet to their head (head direction), the direction from the back perpendicularly outward (vertical back direction), the direction from the subject's body to the left (left direction), the direction from the subject's head to their feet (feet direction), or the direction from the subject's body to the right (right direction), etc.

[0092] It's understandable that CT scanners automatically convert images to a world coordinate system after acquisition. Taking a patient as an example, due to the patient's specific position during CT image acquisition—for instance, the patient's head enters the CT scanner first—the X-axis of the CT image obtained from the CT scan is the left direction, the Y-axis is the vertical direction of the back, and the Z-axis is the head direction. Therefore, the direction vector for the head direction is (0,0,1). Similarly, the direction vector for the vertical direction of the back is (0,1,0), and the direction vector for the left direction is (1,0,0).

[0093] It is understood that the above-mentioned head direction, back vertical direction and left direction are just examples. In actual cases, other directions can also be used to determine the rotation and translation matrix. This application does not limit this.

[0094] 306: The electronic device determines the fourth direction vector of the first reference direction transformed from the ultrasonic coordinate system to the world coordinate system, the fifth direction vector of the second reference direction transformed from the ultrasonic coordinate system to the world coordinate system, and the sixth direction vector of the third reference direction transformed from the ultrasonic coordinate system to the world coordinate system based on the spatial position information of the first ultrasonic image within the tracking range of the optical tracking device; the acquisition time of the first ultrasonic image is the triggering time of the third command.

[0095] In this step, the moment when the user performs the third user operation can be understood as the trigger moment when the ultrasound probe sends the third command. The ultrasound image acquired at the trigger moment of the third command can be understood as the aforementioned first ultrasound image. For electronic devices, due to the short distance and short time delay, the first ultrasound image collected after receiving the third command can be used as the aforementioned first ultrasound image.

[0096] In this embodiment, the transformation from the ultrasonic coordinate system to the world coordinate system is based on an optical tracking device. In this embodiment, the ultrasonic device is equipped with N markers, and the transformation matrix between the N markers and the optical tracking device can be obtained from the coordinates of the N markers. Therefore, after the electronic device receives the aforementioned third instruction, it can determine the direction vector of the reference direction in the ultrasonic coordinate system based on the relative positional relationship between the first ultrasonic image acquired at the trigger time of the third instruction and the implementation object. Then, the transformation matrix can be used to obtain the direction vector of the reference direction after transformation from the ultrasonic coordinate system to the world coordinate system.

[0097] Taking a patient as the subject of the study, the first reference direction as the head direction, and the second reference direction vector as the vertical direction of the back as an example, when scanning the patient's kidneys using an ultrasound device along the direction of the spine, the X direction of the ultrasound coordinate system is the same as the head direction, and the Y direction of the ultrasound coordinate system is the same as the vertical direction of the back. Therefore, when the transformation matrix is ​​represented by equation (1), the direction vector of the head direction (first reference direction) in the world coordinate system can be obtained. For (A) 11 A 21 A 31 ), the direction vector of the back vertical direction (second reference direction) For (A) 12 A 22 A 32 Similarly, when the third reference direction is to the left, the left direction, along with the head direction and the extreme value direction of the back, follows the right-hand rule. Therefore, the direction vector of the left direction is... For (A) 13 A 23 A 33 ).

[0098]

[0099] Optionally, when the first reference direction, the second reference direction, and the third reference direction are perpendicular to each other, the computational load is small and the efficiency is high.

[0100] 307: The electronic device determines the rotation and translation matrix based on the first angle and first position offset between the first direction vector and the fourth direction vector, the second angle and second position offset between the second direction vector and the fifth direction vector, and the third angle and third position offset between the third direction vector and the sixth direction vector.

[0101] As can be understood from steps 305 and 306 above, when the patient's posture remains unchanged, the patient's head direction, back vertical direction, and left direction are inherently fixed in the real world. However, due to the different coordinate systems used by the devices, after converting the ultrasound image and CT image to the same world coordinate system, the direction vectors corresponding to the patient's head direction, back vertical direction, and left direction are different, i.e., the posture of the contour is different.

[0102] In this embodiment, the rotation-translation matrix can be understood as a matrix including rotation and translation components. When calculating the rotation-translation matrix, the rotation components can be determined first. For example, a reference direction can be selected for correction first, and then the remaining two reference directions can be corrected. For instance, if the first reference direction is the head direction, the second reference direction vector is the vertical direction of the back, and the third reference direction is the left direction, the head direction can be corrected first based on the first included angle, and then the rotation components of the rotation-translation matrix can be calculated based on the second and third included angles.

[0103] In this embodiment, the positional offset between direction vectors is used to characterize the relative positional relationship between direction vectors. The translation component of the rotation and translation matrix can be determined by the relative positional relationship between direction vectors.

[0104] 308: The electronic device uses a rotation and translation matrix to transform the initial CT image to obtain the acquired CT image.

[0105] It is understandable that ultrasound images and CT images are obtained from different devices. After converting both ultrasound and CT images to the world coordinate system, their relative spatial positions are uncertain (e.g., they may be some distance apart and at different angles). This embodiment calculates a rotation and translation matrix using a common reference direction between the spaces corresponding to the ultrasound and CT devices, and then transforms the CT image. This allows the CT and ultrasound images in the world coordinate system to be more aligned in position, thereby improving the registration efficiency of subsequent registration using ultrasound and CT images. Furthermore, as long as the position of the object being operated on does not change during the procedure (small variations are permissible), the rotation and translation matrix obtained in this embodiment can be reused.

[0106] In this embodiment, the ultrasound probe includes a button assembly. During ultrasound image scanning, when the ultrasound probe detects a target organ, the user can directly perform a first user operation on the button assembly to send a first command to the electronic device. This command causes the electronic device to use the target ultrasound image acquired between the triggering time of the first command and the triggering time of the second command to reconstruct the contour of the target organ. This process continues until the electronic device receives a second command from the ultrasound probe, thus obtaining the ultrasound contour of the target organ. Then, registration is performed based on the ultrasound contour and the CT contour of the target organ to obtain the registration result. Throughout this reconstruction and registration process, the target ultrasound image used for reconstruction is controlled by the button assembly on the ultrasound probe. This means the user only needs to use the button assembly on the ultrasound probe to select a suitable target ultrasound image for ultrasound contour reconstruction, without needing to use the control device of the electronic device (such as an input device like a mouse). This makes the operation simple, convenient, and efficient.

[0107] Optionally, the ultrasound probe provided in this application embodiment can also be used in the needle insertion point determination stage, which is later than the image registration stage. In some embodiments, after the electronic device performs registration based on the ultrasound profile and CT profile of the target organ and obtains the registration result, step 303 further includes:

[0108] 309: When the target point is scanned using an ultrasonic probe, in response to a fourth user operation on the button assembly, the ultrasonic probe sends a fourth instruction to the electronic device, the fourth instruction indicating the selected needle insertion point; accordingly, the electronic device receives the fourth instruction.

[0109] After the preceding image registration stage, the target point can be accurately displayed in the world coordinate system (e.g., the actual location of the lesion). Therefore, the user can use the ultrasound probe to scan the object and observe and determine whether the target point has been scanned through the monitor. It can be understood that when the ultrasound probe performs an ultrasound scan, the scanning range is a "plane." Therefore, multiple scans can be performed by changing the position of the ultrasound probe. If the target point is displayed on the monitor, it can be considered that the ultrasound probe has scanned the target point.

[0110] 310: The electronic device uses the coordinates of the target point on the ultrasonic probe at the first moment as the coordinates of the needle insertion point. The target point is located on the line segment where the scanning surface of the ultrasonic probe intersects with the ultrasonic probe. The first moment is the triggering moment of the fourth command.

[0111] In this embodiment, the target point is located on the line segment where the scanning surface of the ultrasonic probe intersects with the ultrasonic probe. For easier understanding, please refer to [link to relevant documentation]. Figure 4 , Figure 4This is a schematic diagram of an ultrasonic scan using an ultrasonic probe, provided in an embodiment of this application.

[0112] like Figure 4 As shown, the line segment where the scanning surface 402 intersects with the ultrasound probe 401 can be understood as line segment AC. The target point mentioned above can be any point on line segment AC, such as point A, point B, or point C.

[0113] Since the communication time difference between signaling commands is negligible, the triggering time of the fourth command can be considered to be the same as the receiving time of the fourth command by the electronic device. Therefore, the electronic device can obtain the coordinates of the aforementioned N markers upon receiving the fourth command. The obtained coordinates can be understood as the coordinates of the N markers at the first moment. Since the N markers on the ultrasonic probe are within the tracking range of the optical tracking device, the coordinates of the target point can be obtained through these N markers and the design drawing of the ultrasonic probe, thereby determining the coordinates of the needle insertion point.

[0114] In this embodiment, compared to other solutions that determine the needle insertion point from CT images or registered images, this application determines the needle insertion point based on an ultrasound probe. When the ultrasound probe scans the target point, the needle insertion point is determined based on the line segment where the scanning surface of the ultrasound probe intersects with the probe. This needle insertion point has a higher probability of accurately piercing the target point, thus resulting in a more accurate puncture path. Furthermore, in this solution, after the ultrasound probe scans the target point, the operator can directly select the coordinates of N markers using the button component, and then select the coordinates of the needle insertion point. There is no need to separately operate the electronic control device to select the needle insertion point, making the operation simple and convenient.

[0115] In this embodiment, after determining the needle insertion point, the puncture path can be determined based on the needle insertion point. Once the puncture path is determined, the puncture stage can be considered to have commenced. In this embodiment, during the puncture stage, the electronic device can overlay the planned puncture path onto the ultrasound image in real time, allowing the operator to adjust the puncture needle. In some embodiments, after step 310, which uses the coordinates of the target point on the ultrasound probe at the first moment as the coordinates of the needle insertion point, the method further includes:

[0116] 311: The electronic device determines the first puncture path based on the needle entry point and the target point; the first puncture path includes the needle entry point and the target point;

[0117] Based on the principle that two points determine a straight line, and given the determined needle insertion point and target point, a linear puncture path, i.e., the first puncture path, can be determined. It is understood that, since a holding needle is used for puncture in practice, and the holding needle itself has a certain thickness, the aforementioned first puncture path can be understood as a cylinder with a diameter less than or equal to a certain threshold. This threshold can be set according to actual circumstances, and this application does not impose any limitations on it.

[0118] 312: The electronic device determines the first plane in which the currently acquired second ultrasound image is located.

[0119] It is understandable that when using an ultrasound probe to scan an object, ultrasound images can be obtained in real time. The ultrasound images acquired by the ultrasound equipment can essentially be understood as two-dimensional images. However, the spatial position information of the ultrasound probe is determined by the coordinates of the aforementioned N markers. The relative positional relationship between the ultrasound image obtained within the scanning range of the ultrasound probe and the ultrasound probe itself is fixed. Therefore, the spatial position information of the ultrasound image can also be determined using the coordinates of the aforementioned N markers. All of the above spatial position information is three-dimensional spatial position information.

[0120] In this step, the ultrasound image currently acquired by the electronic device can be understood as the second ultrasound image mentioned above, and the plane where the second ultrasound image is located can be understood as the first plane mentioned above. The first plane is a three-dimensional plane in the world coordinate system, that is, the ultrasound image plane in three dimensions.

[0121] 313: When the electronic device includes the first puncture path on the first plane, the electronic device transforms the first puncture path based on the transformation matrix to obtain the second puncture path; the transformation matrix is ​​determined based on the spatial positions of N markers on the ultrasound probe.

[0122] It is understood that the first puncture path planned based on the needle insertion point and the target point is in the world coordinate system and is fixed. Based on the description of step 311, the first puncture path can be understood as a cylinder with a diameter less than or equal to a threshold. In this embodiment, the first plane including the first puncture path can be understood as the first plane axially cutting the first puncture path (rather than radially cutting it), that is, the first plane including the first puncture path. Since the diameter of the first puncture path is small, the number of pixels obtained after radial cutting will inevitably be much less than the number of pixels obtained after axial cutting. Therefore, the inclusion of the first puncture path on the first plane can be determined by the number of intersecting pixels. If the number of pixels is greater than or equal to the number threshold, it can be considered that the first plane includes the first puncture path; otherwise, it can be considered that the first plane does not include the first puncture path. The number threshold can be set according to the actual situation, and can be set to be greater than the number of pixels obtained after radial cutting of the first puncture path. This application does not limit this.

[0123] For ease of understanding, please refer to the example provided. Figure 5a , Figure 5a This is a schematic diagram of a first plane including a first puncture path provided in an embodiment of this application.

[0124] It is understandable that the N markers on the ultrasonic probe are within the tracking range of the optical tracking device. After the optical tracking device locates the N markers, it can obtain the matrix that transforms the optical coordinate system to the world coordinate system, referred to as the initial transformation matrix T.

[0125] In this step, we solve for the inverse matrix T of the initial transformation matrix. -1 Assuming the data obtained when the first puncture path is included on the first plane is P1, then based on T -1 *P1 obtains the second puncture path described above, that is, changes the spatial position of the first puncture path, transforming the first puncture path into a two-dimensional ultrasound image plane. Wherein, the inverse matrix T... -1 This can be understood as the transformation matrix in this step.

[0126] 314: The electronic device overlays the second puncture path onto the second ultrasound image.

[0127] It is understandable that after the first puncture path is converted to the two-dimensional ultrasound image plane to obtain the second puncture path, the second puncture path can be further displayed on the already displayed ultrasound image, that is, the second puncture path can be superimposed.

[0128] In some embodiments, the second puncture path includes a modified needle insertion point and a modified target point; step 314, the electronic device overlays the second puncture path onto the second ultrasound image, including:

[0129] 3141: The electronic device overlays the second puncture path, as well as the modified needle insertion point and the modified target point, onto the second ultrasound image.

[0130] It is understandable that the needle insertion point and the target point are located on the first puncture path. When the first puncture path is transformed using a transformation matrix, the needle insertion point and the target point will also be transformed accordingly. This embodiment visually displays the transformed needle insertion point and the transformed target point, allowing the operator to observe them more intuitively and adjust the position or orientation of the needle. For example, the transformed needle insertion point and the transformed target point can be displayed using a different color than other points.

[0131] For example, please refer to Figure 5b , Figure 5b This is a schematic diagram showing the puncture path provided in an embodiment of this application.

[0132] like Figure 5b As shown, puncture path 503 can be understood as the aforementioned second puncture path, which includes a modified needle insertion point 501 and a modified target point 502. Path 504 can be understood as the path obtained by ultrasound scanning after the puncture needle is inserted into the target. It can be understood that during the puncture process, the operator can use the path 504 of the inserted puncture needle (e.g., ...) to guide the puncture. Figure 5b The needle tip of the puncture needle (before reaching the target point) is compared with the planned puncture path 503, and the puncture needle is adjusted accordingly.

[0133] The methods provided in the embodiments of this application have been described in detail above. The apparatus provided in the embodiments of this application is described below.

[0134] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 6 The electronic device 60 shown can be the aforementioned image registration device based on an ultrasound probe, which can be applied to a surgical navigation system. The surgical navigation system includes the aforementioned electronic device 60, an optical tracking device, and an ultrasound device. The ultrasound probe of the ultrasound device includes a button assembly, and at least N markers are provided on the ultrasound probe. These N markers are within the tracking range of the optical tracking device, which positions the ultrasound probe by tracking the N markers, where N is an integer greater than or equal to 3. Exemplarily, the electronic device 60 can be the aforementioned electronic device 101, used to implement the method provided in the embodiments of this application.

[0135] The electronic device 60 includes a communication unit 600, a reconstruction unit 601, and a registration unit 602. Optionally, it may also include a determination unit 603, a conversion unit 604, and a display unit 605. The descriptions of each unit are as follows:

[0136] The communication unit 600 is used to receive a first instruction sent by the ultrasound probe when the ultrasound probe scans the target organ. The first instruction is used to indicate the start of image acquisition and reconstruction. The first instruction is triggered by a first user operation on the button assembly.

[0137] The reconstruction unit 601 is used to reconstruct multiple target ultrasound images acquired by the ultrasound device starting from the moment the first instruction is received, until the second instruction sent by the ultrasound probe is received, and to obtain the ultrasound contour of the target organ; the second instruction is triggered by a second user operation on the button assembly, and the second instruction is used to indicate the end of image acquisition and reconstruction, and the acquisition time of the target ultrasound image is between the triggering time of the first instruction and the triggering time of the second instruction.

[0138] The registration unit 602 is used to register the target organ based on the ultrasound profile and the computed tomography (CT) profile of the target organ to obtain the registration result. The CT profile of the target organ is segmented from the obtained CT image.

[0139] Optionally, the communication unit 600 is also configured to receive a third instruction sent by the ultrasound probe, the third instruction being used to instruct position calibration, the third instruction being triggered by a third user operation on the button assembly.

[0140] The determining unit 603 is used to determine the first reference direction transformed from the CT coordinate system to the world coordinate system as a first direction vector, the second reference direction transformed from the CT coordinate system to the world coordinate system as a second direction vector, and the third reference direction transformed from the CT coordinate system to the world coordinate system as a third direction vector; the first reference direction, the second reference direction, and the third reference direction are common to the space corresponding to the ultrasound equipment and the space corresponding to the CT equipment, and the CT coordinate system is the coordinate system corresponding to the CT equipment;

[0141] The determining unit 603 is further configured to determine, based on the spatial position information of the first ultrasonic image within the tracking range of the optical tracking device, the fourth direction vector of the first reference direction transformed from the ultrasonic coordinate system to the world coordinate system, the fifth direction vector of the second reference direction transformed from the ultrasonic coordinate system to the world coordinate system, and the sixth direction vector of the third reference direction transformed from the ultrasonic coordinate system to the world coordinate system; the ultrasonic coordinate system is the coordinate system corresponding to the ultrasonic device, and the acquisition time of the first ultrasonic image is the triggering time of the third command;

[0142] The determining unit 603 is further configured to determine the rotation and translation matrix based on the first angle and first position offset between the first direction vector and the fourth direction vector, the second angle and second position offset between the second direction vector and the fifth direction vector, and the third angle and third position offset between the third direction vector and the sixth direction vector.

[0143] The conversion unit 604 is used to transform the initial CT image using the aforementioned rotation and translation matrix to obtain the previously obtained CT image.

[0144] Optionally, the communication unit 600 is further configured to receive a fourth instruction sent by the ultrasound probe when the target point is scanned using the ultrasound probe. The fourth instruction is used to indicate the selection of the needle insertion point. The fourth instruction is triggered by a fourth user operation on the button assembly. The needle insertion point determination stage is later than the image registration stage.

[0145] The determining unit 603 is further configured to use the coordinates of the target point on the ultrasound probe at the first moment as the coordinates of the needle insertion point. The target point is located on the line segment where the scanning surface of the ultrasound probe intersects with the ultrasound probe. The first moment is the triggering moment of the fourth instruction.

[0146] Optionally, the determining unit 603 is further configured to determine a first puncture path based on the needle insertion point and the target point; the first puncture path includes the needle insertion point and the target point.

[0147] The determining unit 603 is also used to determine the first plane corresponding to the currently acquired second ultrasound image;

[0148] The conversion unit 604 is used to convert the first puncture path based on a transformation matrix to obtain a second puncture path when the first plane includes the first puncture path; the transformation matrix is ​​determined based on the spatial positions of N markers on the ultrasound probe.

[0149] Display unit 605 is used to overlay the second puncture path onto the second ultrasound image.

[0150] Optionally, the display unit 605 is also used to overlay the second puncture path, the transformed needle insertion point, and the transformed target point onto the second ultrasound image.

[0151] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this application. For example... Figure 7 The illustrated electronic device 70 includes a memory 701 and a processor 702. Optionally, the electronic device 70 may also include a communication interface 703 and a bus 704; further optionally, the electronic device 70 may also include a display screen 705. The memory 701, processor 702, communication interface 703, and display screen 705 are interconnected via the bus 704.

[0152] The memory 701 provides storage space, which can store data such as the operating system and computer programs. The memory 701 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM).

[0153] The processor 702 is a module that performs arithmetic and logical operations, and can be one or a combination of processing modules such as a central processing unit (CPU), a graphics processing unit (GPU), or a microprocessor unit (MPU). Additionally, the memory 701 stores computer programs, and the processor 702 can call the computer programs stored in the memory 701 to execute corresponding methods.

[0154] The display screen 705 is used to implement the display function of the electronic device 70. For example, the display screen 705 can be used to display ultrasound images, target points, needle insertion points, and puncture paths, etc.

[0155] In the embodiments of this application, such as Figure 7 When the electronic device 70 shown executes the above method, the processor 702 can control the display function of the display screen 705 and the data communication function of the communication interface 703.

[0156] In some embodiments, the processor 702 can be used to implement the functions of the reconstruction unit 601, registration unit 602, determination unit 603, and conversion unit 604 in the electronic device 60. The display screen 705 can be controlled by the processor 702 to implement the functions of the display unit 605 in the electronic device 60. Optionally, data acquired by the communication unit 600 in the electronic device 60 can be obtained through the communication interface 703.

[0157] This application also provides a computer-readable storage medium storing computer code that, when executed on a computer, causes the computer to perform the methods described in the above embodiments.

[0158] This application also provides a computer program product, which includes computer code or a computer program that, when run on a computer, causes the methods described in the above embodiments to be executed.

[0159] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the above claims.

Claims

1. An image registration method based on an ultrasonic probe, characterized in that, The method is applied to an electronic device in a surgical navigation system, the surgical navigation system including the electronic device, an optical tracking device, a display, and an ultrasound device; the ultrasound probe of the ultrasound device includes a button assembly, and at least N markers are disposed on the ultrasound probe, the N markers being within the tracking range of the optical tracking device, the optical tracking device locating the ultrasound probe by tracking the N markers, where N is an integer greater than or equal to 3; the method includes: When the ultrasound probe scans the target organ, a first instruction sent by the ultrasound probe is received. The first instruction is used to indicate the start of image acquisition and reconstruction. The first instruction is triggered by a first user operation on the button assembly. Starting from the moment the first instruction is received, multiple target ultrasound images acquired by the ultrasound device are reconstructed until a second instruction sent by the ultrasound probe is received, thereby obtaining the ultrasound contour of the target organ. The second instruction is triggered by a second user operation on the button assembly, and the second instruction is used to indicate the end of image acquisition and reconstruction. The acquisition time of the target ultrasound images is between the triggering time of the first instruction and the triggering time of the second instruction. During the reconstruction process, the ultrasound contour of the target organ is displayed on the display, wherein, during the acquisition time, the ultrasound contour of the target organ changes as the ultrasound probe moves. Registration is performed based on the ultrasound profile and the computed tomography (CT) profile of the target organ to obtain the registration result. The CT profile of the target organ is segmented from the obtained CT images. The method further includes: When the target point is scanned using the ultrasonic probe, a fourth instruction sent by the ultrasonic probe is received. The fourth instruction is used to indicate the selected needle insertion point. The fourth instruction is triggered by a fourth user operation on the button assembly. The coordinates of the target point on the ultrasound probe at the first moment are used as the coordinates of the needle insertion point. The target point is located on the line segment where the scanning surface of the ultrasound probe intersects with the ultrasound probe. The first moment is the triggering moment of the fourth command. After using the coordinates of the target point on the ultrasonic probe at the first moment as the coordinates of the needle insertion point, the method further includes: A first puncture path is determined based on the needle insertion point and the target point; the first puncture path includes the needle insertion point and the target point. Determine the first plane corresponding to the currently acquired second ultrasound image; When the first puncture path is included on the first plane, the first puncture path is transformed based on a transformation matrix to obtain a second puncture path; the transformation matrix is ​​determined based on the spatial positions of N markers on the ultrasound probe. The second puncture path is superimposed on the second ultrasound image; The second puncture path includes a modified needle insertion point and a modified target point; the superimposition of the second puncture path on the second ultrasound image includes: The second puncture path, the transformed needle insertion point, and the transformed target point are superimposed on the second ultrasound image. The transformed needle insertion point and the transformed target point are displayed in a different way than other points on the second puncture path.

2. The method according to claim 1, characterized in that, Before receiving the first instruction sent by the ultrasound probe, the method further includes: Receive a third instruction sent by the ultrasonic probe, the third instruction being used to instruct position calibration, the third instruction being triggered by a third user operation on the button assembly; A first reference direction is determined by transforming a first direction vector from the CT coordinate system to the world coordinate system, a second reference direction is determined by transforming a second direction vector from the CT coordinate system to the world coordinate system, and a third reference direction is determined by transforming a third direction vector from the CT coordinate system to the world coordinate system; the first reference direction, the second reference direction, and the third reference direction are common to the space corresponding to the ultrasound device and the space corresponding to the CT device, and the CT coordinate system is the coordinate system corresponding to the CT device; Based on the spatial position information of the first ultrasound image within the tracking range of the optical tracking device, a fourth direction vector is determined from the ultrasound coordinate system to the world coordinate system for the transformation of the first reference direction, a fifth direction vector is determined from the ultrasound coordinate system to the world coordinate system for the transformation of the second reference direction, and a sixth direction vector is determined from the ultrasound coordinate system to the world coordinate system for the transformation of the third reference direction; the ultrasound coordinate system is the coordinate system corresponding to the ultrasound device, and the acquisition time of the first ultrasound image is the trigger time of the third command; Based on the first angle and first position offset between the first direction vector and the fourth direction vector, the second angle and second position offset between the second direction vector and the fifth direction vector, and the third angle and third position offset between the third direction vector and the sixth direction vector, the rotation and translation matrix is ​​determined; The rotation and translation matrix is ​​used to transform the initial CT image to obtain the acquired CT image.

3. A surgical navigation system, characterized in that, The surgical navigation system includes an electronic device, an optical tracking device, a display, and an ultrasound device; the ultrasound probe of the ultrasound device includes a button assembly, and at least N markers are provided on the ultrasound probe, the N markers being within the tracking range of the optical tracking device, and the optical tracking device locating the ultrasound probe by tracking the N markers; where N is an integer greater than or equal to 3; The electronic device is configured to receive a first instruction sent by the ultrasound probe when the ultrasound probe scans a target organ. The first instruction is used to indicate the start of image acquisition and reconstruction. The first instruction is triggered by a first user operation on the button assembly. The electronic device is configured to reconstruct multiple target ultrasound images acquired by the ultrasound device, starting from the moment the first instruction is received, until a second instruction is received from the ultrasound probe, thereby obtaining the ultrasound contour of the target organ; the second instruction is triggered by a second user operation on the button assembly, and the second instruction is used to indicate the end of image acquisition and reconstruction; the acquisition time of the target ultrasound images is between the triggering time of the first instruction and the triggering time of the second instruction; during the reconstruction process, the ultrasound contour of the target organ is displayed on the display, wherein, during the acquisition time, the ultrasound contour of the target organ changes with the movement of the ultrasound probe; The electronic device is used to register the ultrasound profile of the target organ and the computed tomography (CT) profile of the target organ to obtain a registration result, wherein the CT profile of the target organ is segmented from the obtained CT image. The electronic device is used to receive a fourth instruction sent by the ultrasonic probe when the target point is scanned by the ultrasonic probe. The fourth instruction is used to indicate the selection of the needle insertion point. The fourth instruction is triggered by a fourth user operation on the button assembly. The electronic device is used to take the coordinates of the target point on the ultrasonic probe at a first moment as the coordinates of the needle insertion point. The target point is located on the line segment where the scanning surface of the ultrasonic probe intersects with the ultrasonic probe. The first moment is the triggering moment of the fourth command. The electronic device is used to determine a first puncture path based on the aforementioned needle insertion point and the aforementioned target point; the first puncture path includes the aforementioned needle insertion point and the aforementioned target point. The electronic device is used to determine the first plane corresponding to the currently acquired second ultrasound image; The electronic device is used to transform the first puncture path based on a transformation matrix to obtain a second puncture path when the first plane includes the first puncture path; the transformation matrix is ​​determined based on the spatial positions of N markers on the ultrasound probe. The electronic device is used to overlay and display the second puncture path on the second ultrasound image; The second puncture path includes a modified needle insertion point and a modified target point; the electronic device overlays the second puncture path onto the second ultrasound image, including: overlaying the second puncture path, the modified needle insertion point, and the modified target point onto the second ultrasound image, wherein the display method of the modified needle insertion point and the modified target point is different from the display method of other points on the second puncture path.

4. An image registration device based on an ultrasonic probe, characterized in that, Includes units for performing the method as described in any one of claims 1-2.

5. An electronic device, characterized in that, include: A processor and a memory, wherein the memory stores a computer program, and the processor invokes the computer program stored in the memory to perform the method as described in any one of claims 1-2.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on one or more processors, causes the method as described in any one of claims 1-2 to be performed.

Images