A method for ultrasonic Doppler imaging of large targets

Abstract

The present invention discloses a method for ultrasonic Doppler imaging of large targets, including: Step 1, dividing the target into multiple segments according to the length of the large target; Step 2, using a camera to record the position change of the ultrasonic imaging probe and using a slide rail to track the movement of the probe; Step 3, transforming all the collected images into the same coordinate system according to the position relationship between adjacent segments; Step 4, recording the position relationship between each frame of image and the camera when collecting the ultrasonic Doppler image of each segment; Step 5, mapping the collected images into a three-dimensional space through coordinate transformation to perform three-dimensional ultrasonic Doppler imaging. The present invention utilizes a convenient and low-cost optical positioning three-dimensional ultrasonic Doppler imaging system to realize a method for ultrasonic Doppler imaging of large targets, and solves the problem of limited imaging range of the camera in general optical positioning systems.

Description

Technical Field

[0001] The present invention relates to a method for ultrasonic Doppler imaging, in particular to a method for ultrasonic Doppler imaging of large targets. Background Art

[0002] Compared with ultrasonic imaging, ultrasonic Doppler imaging can additionally observe dynamic functional information such as the flow velocity and direction of a target. However, traditional two-dimensional imaging can only observe its cross-sectional view, and three-dimensional ultrasonic Doppler imaging can better meet the observation requirements for complex targets. Three-dimensional ultrasonic imaging is a new type of ultrasonic imaging technology. Imaging with a hand-held probe through optical positioning is a relatively convenient and low-cost method. Using a single-line one-dimensional ultrasonic probe, multiple two-dimensional scans are performed, and an optical positioning system is used to obtain the position of each two-dimensional scan. Multiple two-dimensional images are reconstructed into a three-dimensional ultrasonic image. However, due to the limitation of the imaging range of the camera, the target range for three-dimensional reconstruction using optical positioning is limited. Summary of the Invention

[0003] Object of the Invention: The technical problem to be solved by the present invention is to provide a method for ultrasonic Doppler imaging of large targets in view of the deficiencies of the prior art.

[0004] To solve the above technical problem, the present invention discloses a method for ultrasonic Doppler imaging of large targets, including the following steps:

[0005] Step 1, divide the target into multiple segments according to the length of the large target;

[0006] Step 2, use a camera to record the position change of the ultrasonic imaging probe, and use a slide rail to track the movement of the probe;

[0007] Step 3, transform all the collected images into the same coordinate system according to the position relationship between adjacent two segments;

[0008] Step 4, record the position relationship between each frame of image and the camera when collecting ultrasonic Doppler images of each segment;

[0009] Step 5, map the collected images into a three-dimensional space through coordinate transformation to perform three-dimensional ultrasonic Doppler imaging.

[0010] In the present invention, Step 1 includes:

[0011] Divide the target into several segments according to the imaging range of the camera and the position relationship between the target and the camera, and ensure that after the camera rotates and translates, each segment of the target is within the imaging range of the camera; assume that the target is divided into k segments, and each segment is denoted as 1, 2,..., k.

[0012] In Step 2 of the present invention, the method for using the slide rail to track the movement of the probe includes:

[0013] Fix the camera on the slider of the slide rail, and control the movement of the slider through the motor and the motion controller; use the camera to track the movement of the ultrasonic probe. When it is detected that the probe moves out of the camera's field of view, a signal is sent to the motion controller, and the slider on the slide rail starts to move, driving the camera to slide to achieve automatic tracking; the motion controller provides a communication protocol, and through programming, the communication between the computer and the motion controller is realized, associating the movement of the ultrasonic probe with the movement of the camera on the slide rail to form a feedback system for camera positioning and tracking.

[0014] The method of using the camera to record the position change of the ultrasonic imaging probe in step 2 includes:

[0015] A calibration board is fixed on the probe detected by the camera. Through the Zhang Zhengyou chessboard calibration method, the position relationship of the calibration board relative to the camera is obtained, and the number of feature points detected on the calibration board is recorded to determine whether the calibration board moves out of the camera's field of view; record the initial position of the camera on the slide rail as P 0 , use the triangulation method. According to the internal parameters of the camera, the actual size of the calibration board, and the size of the calibration board on the camera imaging plane, calculate the distance between the camera and the probe; use the field of view angle of the camera and the distance between the camera and the probe to calculate the distance the camera needs to move, and then select the moving distance according to the pre-calibrated data, and record the corresponding position as P j , and send a signal to the motion controller.

[0016] Step 2 further includes:

[0017] Calibrate the position relationship between the probe and the calibration board in advance, and record the corresponding rotation and translation matrix as R i2w and t i2w ; according to the size of the target and the imaging range of the camera, set multiple sets of moving distances and perform calibration, that is, the position change of the camera each time during acquisition.

[0018] In step 2, when calibrating the predetermined position of the camera, two adjacent camera poses are taken as a group, and the same calibration board is calibrated respectively, and the corresponding rotation and translation matrix is recorded; record the camera position corresponding to the jth segment of the target as P j , j < k; the rotation and translation matrix is R j and t j , place the calibration board to ensure that when the camera moves to the P j+1 position, the calibration board can be located, and the corresponding rotation and translation matrix is R j+1 and t j+1 .

[0019] In the present invention, step 3 includes:

[0020] Record a point in the camera coordinate system at the P j+1 position as [xj+1 , y j+1 , z j+1 T , transform it to the camera coordinate system at position P j . The method is as follows:

[0021]

[0022] Transform the points in the camera coordinate system of each segment to the camera coordinate system at position P 1 .

[0023] In the present invention, step 4 includes:

[0024] When collecting ultrasonic Doppler images of each segment of the target respectively, keep the position of the camera fixed, fix a calibration board on the probe, pre-calibrate the position relationship between the probe and the calibration board, hold the probe and slowly move it along the surface of the target, collect the image of the calibration board through the camera, and use the visual calibration algorithm to locate the position relationship between the probe and the camera during the movement.

[0025] In the present invention, step 4 further includes:

[0026] When the ultrasonic probe is imaging, collect each calibration board photo in real time, perform calibration to obtain the rotation and translation matrix of the calibration board relative to the camera; when imaging the j-th segment of the target, denote the rotation and translation matrix of the calibration board corresponding to the i-th image collected as and Taking the upper left pixel of the ultrasonic Doppler image as the coordinate origin, the horizontal direction as the x-axis, the vertical direction as the y-axis, the image plane as the XOY plane, and the direction perpendicular to the image plane as the Z-axis to establish a three-dimensional coordinate system, then the coordinates of each point on the image are [u, v, 0] T , and denote the coordinates after transformation to the corresponding camera coordinate system as [x c , y c , z c T , according to the position relationship R i2w and t i2w between the probe and the calibration board, then:

[0027]

[0028] In the present invention, step 5 includes:

[0029] According to the position transformation relationship between the adjacent position camera coordinate systems described in step 3, and the position relationship between the ultrasonic Doppler imaging plane coordinate system and the camera coordinate system described in step 4, transform all the collected pixel points to the initial camera coordinate system, map the plane points to the three-dimensional space, and splice the reconstruction results of each segment of ultrasonic Doppler to realize three-dimensional ultrasonic Doppler imaging of a large target.​​

[0030] Beneficial effects:

[0031] The present invention proposes a method for automatically tracking a target by a camera, segmenting the target into three-dimensional reconstructions and then stitching them together, and performing ultrasonic Doppler imaging by holding an ultrasonic probe. When collecting data, the imaging area can be autonomously selected, which is more conducive to observing the entire picture of the expected target. This fills the gap in the method of ultrasonic Doppler imaging for large targets. Brief description of the drawings

[0032] The following further specifically describes the present invention in conjunction with the drawings and specific embodiments, and the above and / or other advantages of the present invention will become clearer.

[0033] Figure 1 is a schematic flowchart of the present invention.

[0034] Figure 2 is a schematic diagram of segmenting the target according to the camera imaging range.

[0035] Figure 3 is a schematic diagram of the position of the camera and the slide rail.

[0036] Figure 4 is a conversion diagram of the coordinate relationship of the three-dimensional reconstruction. Specific embodiments

[0037] A method for ultrasonic Doppler imaging of a large target includes the following steps:

[0038] Step 1, divide the target into multiple segments according to the length of the large target;

[0039] According to the imaging range of the camera and the positional relationship between the target and the camera, divide the target into several segments, and ensure that after the camera rotates and translates, each segment of the target is within the imaging range of the camera; assume that the target is divided into k segments, and each segment is denoted as 1, 2,..., k.

[0040] Step 2, use the camera to record the position change of the ultrasonic imaging probe, and use the slide rail to track the movement of the probe;

[0041] The method of using the slide rail to track the movement of the probe includes:

[0042] Fix the camera on the slider of the slide rail, and control the movement of the slider through a motor and a motion controller; use the camera to track the movement of the ultrasonic probe. When it is detected that the probe moves out of the camera's field of view, a signal is sent to the motion controller, and the slider on the slide rail starts to move, driving the camera to slide to achieve automatic tracking; the motion controller provides a communication protocol, and through programming, the communication between the computer and the motion controller is realized, associating the movement of the ultrasonic probe with the movement of the camera on the slide rail to form a feedback system for camera positioning and tracking.

[0043] The method of using a camera to record the position change of an ultrasonic imaging probe includes:

[0044] A calibration board is fixed on the probe detected by the camera. Through the Zhang Zhengyou checkerboard calibration method (reference: Z. Zhang, "A flexible new technique for camera calibration," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1330 - 1334, Nov. 2000), the position relationship between the calibration board and the camera is obtained, and the number of feature points detected on the calibration board is recorded to determine whether the calibration board has moved out of the camera's field of view.

[0045] The position relationship between the probe and the calibration board is calibrated in advance, and the corresponding rotation and translation matrix is denoted as R i2w and t i2w ; According to the size of the target and the imaging range of the camera, set the distance that the slider moves each time and perform calibration, that is, the position change of the camera is known each time during actual acquisition.

[0046] When calibrating the predetermined position of the camera, two adjacent camera poses are taken as a group, and the same calibration board is calibrated respectively, and the corresponding rotation and translation matrices are recorded; denote the camera position corresponding to the j - th segment of the target as P j , j < k; the rotation and translation matrices are R j and t j , place the calibration board to ensure that when the camera moves to the P j+1 position, the calibration board can be located, and the corresponding rotation and translation matrices are R j+1 and t j+1 .

[0047] Step 3, according to the position relationship between adjacent two segments, transform all the collected images to the same coordinate system;

[0048] Denote a point in the camera coordinate system at the P j+1 position as [x j+1 , y j+1 , zj+1 T , transform it to the camera coordinate system at position P j . The method is as follows:

[0049]

[0050] Transform the points in each segment of the camera coordinate system to the camera coordinate system at P 1 .

[0051] Step 4, record the position relationship between each frame of the image and the camera when collecting each segment of the ultrasonic Doppler image;

[0052] When collecting the ultrasonic Doppler images of each segment of the target respectively, keep the position of the camera fixed, fix a calibration board on the probe, calibrate the position relationship between the probe and the calibration board in advance, hold the probe and move it slowly along the surface of the target, collect the images of the calibration board through the camera, and use the visual calibration algorithm to locate each position relationship between the probe and the camera during the movement.

[0053] When the ultrasonic probe is imaging, collect each photo of the calibration board in real time, perform calibration, and obtain the rotation and translation matrix of the calibration board relative to the camera; when imaging the j-th segment of the target, record the rotation and translation matrix of the calibration board corresponding to the i-th image collected relative to the camera as and Taking the upper left pixel of the ultrasonic Doppler image as the coordinate origin, the horizontal direction as the x-axis, the vertical direction as the y-axis, the image plane as the XOY plane, and the direction perpendicular to the image plane as the Z-axis to establish a three-dimensional coordinate system, then the coordinates of each point on the image are [u, v, 0] T , and record the coordinates after transformation to the corresponding camera coordinate system as [x c , y c , z c T . According to the position relationship R i2w and t i2w between the probe and the calibration board, then:

[0054]

[0055] Step 5, map the collected images to the three-dimensional space through coordinate transformation to perform three-dimensional ultrasonic Doppler imaging.

[0056] According to the position transformation relationship between the adjacent position camera coordinate systems described in Step 3, and the position relationship between the ultrasonic Doppler imaging plane coordinate system and the camera coordinate system described in Step 4, transform all the pixel points collected to the initial camera coordinate system, map the plane points to the three-dimensional space, and splice the reconstruction results of each segment of the ultrasonic Doppler to achieve three-dimensional ultrasonic Doppler imaging of the large target. ​​

[0057] To make the above objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. An embodiment of the present invention discloses a method for ultrasonic Doppler imaging of large targets.

[0058] A method for ultrasonic Doppler imaging of large targets according to this embodiment, as Figure 1 shown, includes the following steps:

[0059] Step 1, as Figure 2 shown, taking the thigh as the target, the camera is fixed on the slide rail, and ultrasonic Doppler imaging is performed on the blood vessels in the thigh. According to the imaging range of the camera and the positional relationship between the target and the camera, the target is divided into several segments, and it is ensured that after a certain rotation and translation of the camera, each segment of the target can be within the imaging range of the camera.

[0060] Step 2, fix a calibration board on the probe, and pre-calibrate the positional relationship between the probe and the calibration board, and the corresponding rotation and translation matrices are R i2w , t i2w . As Figure 3 shown, fix the camera on the slider of the slide rail, and precisely control the movement of the slider through the motor and the motion controller. According to the size of the target and the imaging range of the camera, pre-set the distance that the slider moves each time, and perform calibration.

[0061] When calibrating the predetermined positions of the camera, take two adjacent camera poses as a group, calibrate the same calibration board respectively, and record the corresponding rotation and translation matrices. Denote the camera position corresponding to the j (j < k) -th segment of the target as P j , the rotation and translation matrices are R j , t j . Place the calibration board at a suitable position to ensure that when the camera moves to the P j+1 position, the calibration board can still be located, and the corresponding rotation and translation matrices are R j+1 , t j+1 .

[0062] By using the visual calibration method, obtain the positional relationship between the calibration board and the camera, and record the number of feature points detected on the calibration board to determine whether the calibration board has moved out of the camera's field of view. When it is detected that the probe has moved out of the camera's field of view, the computer sends a signal to the motion controller, and the slider on the slide rail drives the camera to slide to achieve automatic tracking.

[0063] Step 3, after the device is fixed, when collecting ultrasonic Doppler images of each segment of the target respectively, the position of the camera does not change. Hold the probe and slowly move it along the surface of the target, and record the rotation and translation matrix of each frame of the image relative to the camera in real time.

[0064] Step 4, when imaging the j-th segment of the target, record the rotation and translation matrix of the calibration board corresponding to the i-th image collected relative to the camera as As Figure 4 shown, taking the pixel at the upper left corner of the ultrasonic Doppler image as the coordinate origin, the horizontal direction as the x-axis, the vertical direction as the y-axis, the image plane as the XOY plane, and the direction perpendicular to the image plane as the Z-axis to establish a three-dimensional coordinate system. It can be known that the coordinates of each point on the image are [u, v, 0] T , and record the coordinates after transformation to the corresponding camera coordinate system as [x c , y c , z c T , according to the positional relationship R i2w , t i2w between the probe and the calibration board, there is:

[0065]

[0066] Record a point in the camera coordinate system at the position of P j+1 as [x j+1 , y j+1 , z j+1 T , and transform it to the camera coordinate system at the position of P j , there is:

[0067]

[0068] According to this iterative relationship, the pixel points in each ultrasonic image can be transformed to the camera coordinate system at the position of P 1 to achieve three-dimensional ultrasonic Doppler reconstruction of large targets.

[0069] The present invention provides an idea and method for ultrasonic Doppler imaging of large targets. There are many methods and ways to specifically implement this technical solution. The above description is only the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention. Each component not clearly defined in this embodiment can be implemented by existing technologies.​​

Claims

1. A method for ultrasonic Doppler imaging of large targets, characterized in that, it includes the following steps: Step 1: Divide the target into multiple segments according to the length of the large target; Step 2: Use a camera to record the position change of the ultrasonic imaging probe, and use a slide rail to track the movement of the probe; Step 3: According to the position relationship between adjacent two segments, transform all the collected images into the same coordinate system; Step 4: Record the position relationship between each frame of image and the camera when collecting the ultrasonic Doppler image of each segment; Step 5: Map the collected images into a three-dimensional space through coordinate transformation to perform three-dimensional ultrasonic Doppler imaging; Among them, in Step 2, the method of using a camera to record the position change of the ultrasonic imaging probe includes: A calibration board is fixed on the probe detected by the camera. Through the Zhang-Zhengyou checkerboard calibration method, the positional relationship between the calibration board and the camera is obtained, and the number of feature points detected on the calibration board is recorded to determine whether the calibration board has moved out of the camera's view; the initial position of the camera on the slide rail is recorded as P 0 , using the triangulation ranging method, according to the internal parameters of the camera, the actual size of the calibration board, and the size of the calibration board on the camera imaging plane, the distance between the camera and the probe is calculated; the distance that the camera needs to move is calculated using the field of view angle of the camera and the distance between the camera and the probe, and then according to the pre-calibrated data, the moving distance is selected, and the corresponding position is recorded as P j , send a signal to the motion controller; In step 2, when calibrating the predetermined positions of the cameras, every two adjacent camera poses form a group. The calibration is performed on the same calibration board respectively, and the corresponding rotation and translation matrices are recorded. Denote the camera position corresponding to the j-th segment of the target as P j , where j < k; the rotation and translation matrices are R j and t j . Place the calibration board to ensure that when the camera moves to the position of P j+1 , the calibration board can be located. The corresponding rotation and translation matrices are R j+1 and t j+1 . Step 3 includes: Record a point in the camera coordinate system at position P j+1 as [x j+1 , y j+1 , z j+1 , T and transform it to the camera coordinate system at position P j by the following method: Transform the points in the camera coordinate system of each segment to the camera coordinate system at P 1 ; Step 4 includes: During the imaging of the ultrasonic probe, each calibration plate photo is collected in real time for calibration to obtain the rotation and translation matrix of the calibration plate relative to the camera. When imaging the j-th segment of the target, the rotation and translation matrix of the calibration plate corresponding to the i-th image collected is denoted as and Taking the upper left pixel of the ultrasonic Doppler image as the coordinate origin, the horizontal direction as the x-axis, the vertical direction as the y-axis, the image plane as the XOY plane, and the direction perpendicular to the image plane as the Z-axis to establish a three-dimensional coordinate system, then the coordinates of each point on the image are [u, v, 0] T , and the coordinates after transformation to the corresponding camera coordinate system are denoted as [x c , y c , z c T , according to the positional relationship R i2w and t i2w between the probe and the calibration plate, then:​ 2. The method for ultrasonic Doppler imaging of large targets according to claim 1, characterized in that, Step 1 includes: According to the imaging range of the camera and the position relationship between the target and the camera, divide the target into several segments, and ensure that after the camera rotates and translates, each segment of the target is within the imaging range of the camera; assume that the target is divided into k segments, and each segment is denoted as 1, 2,..., k.

3. The method for ultrasonic Doppler imaging of large targets according to claim 2, characterized in that, In Step 2, the method of using a slide rail to track the movement of the probe includes: Fix the camera on the slider of the slide rail, and control the movement of the slider through a motor and a motion controller; use the camera to track the movement of the ultrasonic probe. When it is detected that the probe moves out of the camera's view, send a signal to the motion controller, and the slider on the slide rail starts to move, driving the camera to slide to achieve automatic tracking; the motion controller provides a communication protocol, and through programming, realize the communication between the computer and the motion controller, and associate the movement of the ultrasonic probe with the movement of the camera on the slide rail to form a feedback system for camera positioning and tracking.

4. The method for ultrasonic Doppler imaging of large targets according to claim 3, characterized in that, Step 2 further includes: Pre-calibrate the positional relationship between the probe and the calibration board, and record the corresponding rotation and translation matrix as R i2w and t i2w ; According to the size of the target and the imaging range of the camera, set multiple groups of moving distances and perform calibration, that is, record the position change of the camera each time during the acquisition.

5. The method for ultrasonic Doppler imaging of large targets according to claim 4, characterized in that, Step 4 includes: When collecting the ultrasonic Doppler images of each segment of the target respectively, keep the position of the camera unchanged, and fix a calibration board on the probe. Calibrate the position relationship between the probe and the calibration board in advance. Hold the probe and slowly move it along the surface of the target. Collect the images of the calibration board through the camera, and use the visual calibration algorithm to locate the position relationship between the probe and the camera during the movement.

6. The method for ultrasonic Doppler imaging of large targets according to claim 5, characterized in that, Step 5 includes: According to the position relationship between adjacent two segments described in Step 3, and the position relationship between each frame of image and the camera described in Step 4, transform all the collected pixel points into the initial camera coordinate system, map the plane points into a three-dimensional space, and splice the reconstruction results of each segment of ultrasonic Doppler to achieve three-dimensional ultrasonic Doppler imaging of large targets.

Images