Ultrasound image electromagnetic navigation automatic registration method, system, device and medium

Abstract

This invention provides an automatic registration and registration method, system, device, and medium for ultrasonic image electromagnetic navigation, comprising: Step S1: acquiring three-dimensional ultrasonic images and three-dimensional spatial data; Step S2: registering and registering the three-dimensional ultrasonic images and three-dimensional spatial data to obtain the registration and registration results of the visual camera coordinates and the ultrasonic images; Step S3: setting up an electromagnetic calibration tool to obtain the electromagnetic coordinates of the electromagnetic calibration tool and the center coordinates of the electromagnetic calibration tool; Step S4: registering and registering the electromagnetic coordinates and the center coordinates to obtain the registration and registration results of the visual camera and the electromagnetic coordinates; Step S5: based on the registration and registration results of the visual camera coordinates and the ultrasonic images, and the registration and registration results of the visual camera and the electromagnetic coordinates, using three-dimensional spatial data as an intermediate transformation, to obtain the registration and registration results of the ultrasonic images and the electromagnetic coordinates. This invention enables a rapid and efficient automated registration scheme based on a small ball calibrating electromagnetic tool, improving the efficiency of the registration algorithm.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, specifically to an automatic registration and registration method, system, device, and medium for ultrasonic imaging electromagnetic navigation. Background Technology

[0002] Ultrasound imaging electromagnetic navigation automatic registration tools and methods can be applied to surgical navigation and positioning in computer-aided diagnosis and treatment systems and robotic surgical planning and control systems. Doctors need to overcome problems such as insufficient experience, inadequate and non-real-time surgical imaging information, and insufficient puncture accuracy and safety. Using low-cost, real-time ultrasound imaging for assisted guidance in interventional minimally invasive diagnosis and treatment can ensure the widespread adoption of the technology and its applicability to a wide range of hospitals.

[0003] Currently, existing technologies generally involve clamping and binding the probe of a sensor with electromagnetic positioning, and calibrating it using the relative offset of the physical space of the rigid binding. For example, the literature "Bi Jintao, Zhang Yongde, Sun Botao. A method and experimental study on puncture navigation of interventional robots based on electromagnetic tracking and ultrasound images [J]. Journal of Instrumentation, 2019, 40(7):253-262." uses the method of directly measuring and inputting parameters through the electromagnetic positioning system to obtain the coordinate transformation relationship between the electromagnetic space and the probe space. The probe space and the ultrasound image space are calibrated using an N-line model to obtain the spatial transformation relationship. Combining the above two transformation relationships, the transformation relationship between the electromagnetic space and the ultrasound image space is obtained, thus completing the ultrasound navigation image registration algorithm. Generally speaking, the parameters of the electromagnetic positioning space and the physical space of the ultrasound probe have shortcomings such as insufficient measurement accuracy, limited binding methods, and cumbersome operation.

[0004] Ultrasound imaging: Images formed by scanning various organs and viscera of the human body using ultrasound waves.

[0005] Registration / calibration: The coordinate mapping relationship between one space and another space.

[0006] Electromagnetic (positioning / navigation) system: A spatial positioning scheme based on weak electromagnetic fields, consisting of a magnetic field generator, a signal receiver, and a data processing and control unit. Tiny electromagnetic induction coil sensors can be implanted inside the device, enabling accurate positioning with an accuracy of less than 1 mm in a certain space.

[0007] Visual camera: A 3D data scanning imaging device that uses depth scan data in a real-world scene.

[0008] N-line model: In a container with a hollowed-out top opening inside a cuboid, multiple sets of thin metal wires pass through the hollowed-out interior in an N-shape on opposite sides, forming a model for calibration. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides an automatic registration and registration method, system, device, and medium for ultrasonic imaging electromagnetic navigation.

[0010] According to the present invention, an automatic registration and registration method, system, device, and medium for electromagnetic navigation of ultrasound images are provided, the solution of which is as follows:

[0011] In a first aspect, an automatic registration and alignment method for electromagnetic navigation of ultrasound images is provided, the method comprising:

[0012] Step S1: Acquire three-dimensional ultrasound images and three-dimensional spatial data;

[0013] Step S2: Register the three-dimensional ultrasound image and the three-dimensional spatial data to obtain the registration and registration results of the visual camera coordinates and the ultrasound image;

[0014] Step S3: Set up the electromagnetic calibration tool and obtain the electromagnetic coordinates and center coordinates of the electromagnetic calibration tool;

[0015] Step S4: Register and align the electromagnetic coordinates and the center coordinates to obtain the registration and alignment results of the visual camera and the electromagnetic coordinates;

[0016] Step S5: Based on the registration and alignment results of the visual camera coordinates and the ultrasound image, as well as the registration and alignment results of the visual camera and the electromagnetic coordinates, the three-dimensional spatial data is used as an intermediate transformation to obtain the registration and alignment results of the ultrasound image and the electromagnetic coordinates.

[0017] Preferably, step S1 specifically includes: acquiring continuous two-dimensional ultrasound images through an ultrasound probe, reconstructing the two-dimensional ultrasound images into three-dimensional ultrasound images, and obtaining the three-dimensional spatial data by scanning with a visual camera.

[0018] Preferably, step S2 includes: extracting surface point cloud data from the three-dimensional spatial data; extracting pixel point cloud data from the three-dimensional ultrasound image; and registering the three-dimensional ultrasound image and the three-dimensional spatial data using the surface point cloud data and the pixel point cloud data.

[0019] Preferably, the pixel point cloud data is obtained by filtering through a grayscale threshold; the three-dimensional ultrasound image is a single-channel grayscale image, the grayscale threshold is a grayscale range, pixel data outside the grayscale range is removed as impurity noise, and pixel data within the range is used as data.

[0020] Preferably, the electromagnetic calibration tool includes a calibration tool body, a holding slot, an electromagnetic sensor, and external wiring for the sensor; the holding slot is formed inside the calibration tool body; the electromagnetic sensor is placed in the holding slot; and the external wiring for the sensor is connected to the electromagnetic sensor.

[0021] Preferably, step S4 specifically includes: placing the electromagnetic calibration tool within the working range of the electromagnetic field, randomly moving the electromagnetic calibration tool, and sampling data at intervals to obtain the electromagnetic coordinates of the electromagnetic calibration tool and the three-dimensional spatial data;

[0022] Extract the spatial point cloud data of the electromagnetic calibration tool from the three-dimensional spatial data, and calculate the center coordinates of the electromagnetic calibration tool by fitting.

[0023] Using the electromagnetic coordinates and the center coordinates as input, the registration and alignment are calculated to obtain the registration and alignment results of the visual camera and the electromagnetic coordinates.

[0024] Preferably, the electromagnetic coordinates of the electromagnetic calibration tool are pre-calibrated to the center position of the electromagnetic calibration tool, and the electromagnetic coordinates are the coordinates of the center of the electromagnetic calibration tool.

[0025] Secondly, an automatic registration and registration system for electromagnetic navigation of ultrasound images is provided, characterized in that it includes:

[0026] Module M1: Acquires three-dimensional ultrasound images and three-dimensional spatial data;

[0027] Module M2: Registers and aligns the three-dimensional ultrasound image and the three-dimensional spatial data to obtain the registration and alignment results of the visual camera coordinates and the ultrasound image;

[0028] Module M3: Set up the electromagnetic calibration tool and obtain the electromagnetic coordinates and center coordinates of the electromagnetic calibration tool;

[0029] Module M4: Registers and aligns the electromagnetic coordinates and the center coordinates to obtain the registration and alignment results of the visual camera and the electromagnetic coordinates;

[0030] Module M5: Based on the registration and alignment results of the visual camera coordinates and the ultrasound image, as well as the registration and alignment results of the visual camera and the electromagnetic coordinates, the three-dimensional spatial data is used as an intermediate transformation to obtain the registration and alignment results of the ultrasound image and the electromagnetic coordinates.

[0031] Thirdly, an apparatus is provided, the apparatus comprising:

[0032] One or more processors;

[0033] Storage device for storing one or more programs.

[0034] When the one or more programs are executed by the one or more processors, the one or more processors perform the steps in the method.

[0035] Fourthly, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, implements the steps of the method.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. This invention introduces a visual camera, which automatically completes the registration and alignment of ultrasound images and electromagnetic coordinate space with the assistance of a specific small ball electromagnetic calibration tool, making it convenient for application in surgical navigation systems.

[0038] 2. This invention indirectly achieves rapid registration and registration of human ultrasound images and electromagnetic coordinates by registering and registering visual cameras with human spatial ultrasound images, and then combining visual cameras with electromagnetic coordinates for registration and registration. It uses an intermediate transformation transition based on the visual coordinate system for automatic registration and registration algorithm, realizing a fast, efficient and automated registration scheme based on a small ball calibration electromagnetic tool, which greatly improves the efficiency of the registration algorithm. Attached Figure Description

[0039] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0040] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0041] Figure 2 This is a schematic diagram of the electromagnetic calibration tool of the present invention.

[0042] Figure label:

[0043] Small sphere 1 Holding tank 2

[0044] Electromagnetic sensor 3 External wiring 4

[0045] The center of the small ball is at position 5. Detailed Implementation

[0046] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0047] This invention provides an automatic registration and registration method for ultrasonic image electromagnetic navigation, referring to... Figure 1 As shown, the method specifically includes:

[0048] Step S1: Acquire three-dimensional ultrasound images and three-dimensional spatial data.

[0049] Specifically, continuous two-dimensional ultrasound images are acquired using an ultrasound probe, and the two-dimensional ultrasound images are reconstructed into three-dimensional ultrasound images; the three-dimensional spatial data is obtained by scanning with a visual camera.

[0050] Step S2: Register and align the 3D ultrasound images and 3D spatial data to obtain the registration and alignment results of the visual camera coordinates and ultrasound images.

[0051] Specifically, step S2 includes: extracting surface point cloud data from the three-dimensional spatial data; extracting pixel point cloud data from the three-dimensional ultrasound image; and registering the three-dimensional ultrasound image and the three-dimensional spatial data using the surface point cloud data and the pixel point cloud data.

[0052] Pixel point cloud data is obtained by filtering through grayscale thresholds; the three-dimensional ultrasound image is a single-channel grayscale image, and the grayscale threshold is the grayscale range. Pixel data outside the grayscale range is removed as impurity noise, and pixel data within the range is used as data.

[0053] Step S3: Set up the electromagnetic calibration tool and obtain its electromagnetic coordinates and center coordinates. In this embodiment, the electromagnetic calibration tool can be any other centrally symmetrical shape, such as a cube or a regular hexahedron. This embodiment uses a sphere, which has relatively higher recognizability.

[0054] Among them, reference Figure 2 As shown, the electromagnetic calibration tool includes a calibration tool body, a container, an electromagnetic sensor, and external wiring for the sensor. The container is located within the calibration tool body; the electromagnetic sensor is placed in the container; and the external wiring for the sensor is connected to the electromagnetic sensor.

[0055] Specifically, step S3 includes: placing the electromagnetic calibration tool within the working range of the electromagnetic field, randomly moving the electromagnetic calibration tool, and sampling data at intervals to obtain the electromagnetic coordinates and three-dimensional spatial data of the electromagnetic calibration tool.

[0056] Extract the spatial point cloud data of the electromagnetic calibration tool from the three-dimensional spatial data, and calculate the center coordinates of the electromagnetic calibration tool by fitting.

[0057] Using electromagnetic coordinates and center coordinates as input, the registration and alignment are calculated to obtain the registration and alignment results of the visual camera and electromagnetic coordinates.

[0058] The electromagnetic coordinates of the electromagnetic calibration tool are pre-calibrated to the center position of the electromagnetic calibration tool, and the electromagnetic coordinates are the coordinates of the center of the electromagnetic calibration tool.

[0059] Step S4: Register and align the electromagnetic coordinates and the center coordinates to obtain the registration and alignment results of the visual camera and the electromagnetic coordinates.

[0060] Step S5: Based on the registration and alignment results of the visual camera coordinates and ultrasound images, and the registration and alignment results of the visual camera and electromagnetic coordinates, the visual camera is used as an intermediate transformation to obtain the registration and alignment results of the ultrasound images and electromagnetic coordinates.

[0061] The present invention will now be described in more detail.

[0062] This invention provides an automatic registration and alignment method for electromagnetic navigation of ultrasound images, referring to... Figure 1 As shown, the specific content is as follows:

[0063] 1. Acquire continuous two-dimensional ultrasound images using an ultrasound probe, and reconstruct the two-dimensional ultrasound images into three-dimensional ultrasound images;

[0064] 2. Three-dimensional ultrasound images can be directly registered and aligned with data scanned by a human visual camera. The visual camera can extract point cloud data of the three-dimensional structure of the human body surface, i.e., three-dimensional spatial data. In the three-dimensional ultrasound image, pixel data of the body surface can be extracted and converted into point cloud data by the difference in grayscale thresholds in the image. Registration is then performed using the point cloud dataset to obtain the registration and alignment results between the visual camera coordinates and the ultrasound image. In this embodiment, the three-dimensional ultrasound image is a single-channel grayscale image. The grayscale threshold is set within a grayscale range; values ​​outside the range are removed as noise, while values ​​within the range are retained as valid data.

[0065] 3. Using a small-ball electromagnetic calibration tool, an electromagnetic sensor is placed at the center of the tool. The ball is placed within the electromagnetic field's operating range, and the ball is moved randomly. The system samples data at intervals to obtain the ball's electromagnetic coordinates. Simultaneously, partial spherical surface data of the ball from the vision camera is obtained (the vision camera has the function of generating three-dimensional point cloud data; here, the spherical surface data refers to the three-dimensional point cloud data). The center position of the ball is calculated by fitting this data. In this embodiment, the spherical surface data is the point cloud recognized by the vision camera. These points can be understood as points on the surface of a sphere in a three-dimensional coordinate system. While the vision camera cannot recognize all the points on the sphere, the center position can be calculated from the partially recognized points. Calculation methods include least-squares error optimization by solving the spherical equation, and fitting the solution using the Random Sample Consensus Algorithm (RANSAC).

[0066] The electromagnetic coordinates of the small ball electromagnetic calibration tool are pre-calibrated to the center of the small ball. The collected electromagnetic coordinates are the coordinates of the center of the small ball. The electromagnetic coordinates of the center of the small ball and the coordinates of the center of the small ball calculated by the vision camera are used as input to calculate the registration and registration, and obtain the registration and registration results of the vision camera and the electromagnetic coordinates.

[0067] 4. By using a visual camera as an intermediate converter, the registration and alignment results of electromagnetic coordinates and ultrasound images can be obtained.

[0068] Reference Figure 2As shown, the small ball electromagnetic calibration tool used in this invention includes a small ball 1, a cylindrical container 2 capable of accommodating an electromagnetic sensor, an electromagnetic sensor 3 placed in the container 2, and an external sensor wiring 4 connected to the sensor 3. The center of the electromagnetic sensor 3 coincides with the center position 5 of the small ball. Preferably, the small ball 1 is pure green to ensure that the position information is clearly captured by the visual camera.

[0069] This invention provides an automatic registration and alignment method, system, device, and medium for ultrasonic image electromagnetic navigation. It indirectly achieves rapid registration and alignment of human ultrasound images and electromagnetic coordinates by registering and aligning a visual camera with human spatial ultrasound images, and then combining this with registration and alignment of the visual camera with electromagnetic coordinates. After continuous ultrasound image scanning, three-dimensional ultrasound image data is reconstructed. The three-dimensional ultrasound images are then registered and aligned with the three-dimensional data from the visual camera to obtain the registration result of the visual camera and ultrasound images. Finally, the registration and alignment of electromagnetic coordinates and ultrasound images can be obtained through the combined use of the visual camera. This invention avoids the cumbersome calibration process of ultrasound images and scanning probes using N-line model fixtures, and avoids the insufficient accuracy of binding and calibration of ultrasound probes and electromagnetic tools. It prepares for the rapid and automatic completion of system registration and alignment between electromagnetic systems and ultrasound images.

[0070] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0071] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. An automatic registration and registration method for ultrasonic imaging electromagnetic navigation, characterized in that, include: Step S1: Acquire three-dimensional ultrasound images and three-dimensional spatial data; Step S2: Register the three-dimensional ultrasound image and the three-dimensional spatial data to obtain the registration and registration results of the visual camera coordinates and the ultrasound image; Step S3: Set up the electromagnetic calibration tool and obtain the electromagnetic coordinates and center coordinates of the electromagnetic calibration tool; Step S4: Register and align the electromagnetic coordinates and the center coordinates to obtain the registration and alignment results of the visual camera and the electromagnetic coordinates; Step S5: Based on the registration and alignment results of the visual camera coordinates and the ultrasound image, as well as the registration and alignment results of the visual camera and the electromagnetic coordinates, the three-dimensional spatial data is used as an intermediate transformation to obtain the registration and alignment results of the ultrasound image and the electromagnetic coordinates.

2. The automatic registration and registration method for ultrasonic imaging electromagnetic navigation according to claim 1, characterized in that, Step S1 specifically includes: acquiring continuous two-dimensional ultrasound images through an ultrasound probe, reconstructing the two-dimensional ultrasound images into three-dimensional ultrasound images, and obtaining the three-dimensional spatial data by scanning with a visual camera.

3. The automatic registration and registration method for ultrasonic imaging electromagnetic navigation according to claim 1, characterized in that, Step S2 includes: extracting surface point cloud data from the three-dimensional spatial data; extracting pixel point cloud data from the three-dimensional ultrasound image; and registering the three-dimensional ultrasound image and the three-dimensional spatial data using the surface point cloud data and the pixel point cloud data.

4. The automatic registration and registration method for ultrasonic imaging electromagnetic navigation according to claim 3, characterized in that, The pixel point cloud data is obtained by filtering through a grayscale threshold; the three-dimensional ultrasound image is a single-channel grayscale image, the grayscale threshold is a grayscale range, the pixel data outside the grayscale range is removed as impurity noise, and the pixel data within the grayscale range is used as data.

5. The automatic registration and registration method for ultrasonic imaging electromagnetic navigation according to claim 1, characterized in that, The electromagnetic calibration tool includes a calibration tool body, a holding slot, an electromagnetic sensor, and external wiring for the sensor; the holding slot is located inside the calibration tool body; the electromagnetic sensor is placed in the holding slot; and the external wiring for the sensor is connected to the electromagnetic sensor.

6. The automatic registration and registration method for ultrasonic imaging electromagnetic navigation according to claim 5, characterized in that, Step S3 specifically includes: placing the electromagnetic calibration tool within the electromagnetic field working range, randomly moving the electromagnetic calibration tool, and sampling data at intervals to obtain the electromagnetic coordinates of the electromagnetic calibration tool and the three-dimensional spatial data; Extract the spatial point cloud data of the electromagnetic calibration tool from the three-dimensional spatial data, and calculate the center coordinates of the electromagnetic calibration tool by fitting. Using the electromagnetic coordinates and the center coordinates as input, the registration and alignment are calculated to obtain the registration and alignment results of the visual camera and the electromagnetic coordinates.

7. The automatic registration and registration method for ultrasonic imaging electromagnetic navigation according to claim 6, characterized in that, The electromagnetic coordinates of the electromagnetic calibration tool are pre-calibrated to the center position of the electromagnetic calibration tool, and the electromagnetic coordinates are the coordinates of the center of the electromagnetic calibration tool.

8. An automatic registration and registration system for ultrasonic imaging electromagnetic navigation, characterized in that, include: Module M1: Acquires three-dimensional ultrasound images and three-dimensional spatial data; Module M2: Registers and aligns the three-dimensional ultrasound image and the three-dimensional spatial data to obtain the registration and alignment results of the visual camera coordinates and the ultrasound image; Module M3: Set up the electromagnetic calibration tool and obtain the electromagnetic coordinates and center coordinates of the electromagnetic calibration tool; Module M4: Registers and aligns the electromagnetic coordinates and the center coordinates to obtain the registration and alignment results of the visual camera and the electromagnetic coordinates; Module M5: Based on the registration and alignment results of the visual camera coordinates and the ultrasound image, as well as the registration and alignment results of the visual camera and the electromagnetic coordinates, the three-dimensional spatial data is used as an intermediate transformation to obtain the registration and alignment results of the ultrasound image and the electromagnetic coordinates.

9. A device, characterized in that, The device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images