An accurate model construction method of an ultrasonic point cloud

Abstract

The application relates to the technical field of model construction, and discloses a precise model construction method of an ultrasonic point cloud, which comprises the following steps: S1: CT image point cloudization, CT scanning is carried out on a patient before operation, blood vessels and cysts are segmented, and point cloud reconstruction is carried out; S2: ultrasonic image point cloudization; S3: initial reconstruction, an ultrasonic probe installed on a mechanical arm is used to scan along a fixed path in a fixed direction; and S4: accurate reconstruction. The application can construct a precise ultrasonic model by twice reconstruction of ultrasonic images, and can avoid the problem that a slight difference in the position of a cyst in a puncture operation will cause a serious medical accident due to the fact that partial volume effect will cause ultrasonic artifacts in the accurate reconstruction process by using a position compensation and contour compensation mode.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model construction, in particular to a precise model construction method of ultrasonic point cloud. BACKGROUND

[0002] Medical image fusion technology is a hot issue of research at home and abroad today. Medical imaging includes X-ray, ultrasound, computed tomography (CT), magnetic resonance (MRI), single photon emission computed tomography (SPECT), positron emission tomography (PET), infrared, digital subtraction (DSA), fluorescence contrast, etc.

[0003] With the rapid development of three-dimensional data acquisition technology, point cloud data collected by laser radar, structured light sensor and stereo camera has been widely applied. Ultrasonic image has volume effect, and the ultrasonic image seen is not completely accurate. For example, when a blood vessel is punctured, the puncture needle may be seen to enter the blood vessel in the ultrasonic image, but in fact it does not. Partial volume effect is more common in small volume low echo or no echo structure, such as blood vessels, especially small blood vessels, which are easily affected by this effect, resulting in errors during puncture. In the case of manual puncture by a doctor, the ultrasonic probe needs to be moved back and forth to confirm from all directions of the probe, and the image changes are continuously observed and judged, which is very low in efficiency. Therefore, a precise model construction method of ultrasonic point cloud is proposed. SUMMARY

[0004] (I) Technical problems solved

[0005] In view of the deficiencies in the prior art, the present application provides a precise model construction method of ultrasonic point cloud, which solves the problem of errors in the existing ultrasonic point cloud puncture, and the problem of low efficiency in the case of manual puncture by a doctor, which needs to move the ultrasonic probe back and forth to confirm from all directions of the probe, and continuously observe and judge the changes of the image.

[0006] (II) Technical solutions

[0007] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0008] A precise model construction method of ultrasonic point cloud, comprising the following steps:

[0009] S1: CT image point cloud, CT scanning is performed on the patient before the operation, and the blood vessels and cysts are segmented and point cloud reconstruction is performed;

[0010] S2: ultrasonic image point cloud, through the ultrasonic probe installed on the mechanical arm constantly scanning, obtain real-time ultrasonic video information, then obtain the attitude information corresponding to the ultrasonic video information, obtain the ultrasonic probe attitude information (x, y, z, gamma, beta, alpha), then match the ultrasonic image and the corresponding attitude information, the recognition of organs in the ultrasonic image and the display of the recognition result sampling, based on the organ contour sampling points obtained by sampling, three-dimensional coordinate change is carried out, the actual position of each contour point in the mechanical arm coordinate system of the picture is obtained, and the sparse points are supplemented based on the contour characteristics, each converted point is drawn into 3D space for visual display, and the coordinate system of the space is set according to the robot coordinate system;

[0011] S3: initial reconstruction, the ultrasonic probe installed on the mechanical arm is scanned in a fixed direction along a fixed path, real-time ultrasonic information is obtained, then the ultrasonic probe scans the cyst from left to right along the direction perpendicular to the patient's skin, since the ultrasonic beam emitted by the probe has a certain thickness, that is, the obtained image is a superimposed image of the echo information within a certain thickness space;

[0012] S4: accurate reconstruction, based on the ultrasonic point cloud model of the initial reconstruction, the accurate reconstruction of the cyst is realized by means of backstepping, and position compensation and contour compensation are carried out during accurate reconstruction.

[0013] As a further scheme of the application, in S1, the patient is subjected to enhanced CT scanning and ultrasonic scanning, and after scanning, the staff manually selects easily recognizable blood vessels or cysts based on the CT scanning results of the patient, organizes the segmentation of the CT image, marks the blood vessels and cysts, and reconstructs the point cloud image.

[0014] Further, in S2, each frame of image collected is subjected to image recognition, and the organ contour of the corresponding model to be established is extracted, and the organ contour is sampled to obtain contour sampling points.

[0015] On the basis of the foregoing scheme, in S2, specifically, the robot coordinate system, the probe coordinate system and the image coordinate system are defined as {B}, {P} and {I} respectively, the conversion matrix of {P} to {B} is The conversion matrix of {I} to {P} is,

[0016] On the basis of the foregoing scheme, in S3, based on the known position of the cyst, the ultrasonic probe is rigidly connected with the mechanical arm, the mechanical arm is controlled to make the ultrasonic probe and the patient's skin form a fixed included angle, and the cyst is scanned along a fixed route.

[0017] As a further scheme of the application, in S3, the thickness of the ultrasonic probe is d, the translation distance of the probe is L, and the two-dimensional position of the scanned image set P is:

[0018]

[0019] Further, the position compensation formula in S4 is Wherein x P ' is the x-axis coordinate of the point set P after accurate reconstruction, when the ultrasonic probe first contacts the edge point of the contour, the point cloud position is constructed according to formula (1), and then the point cloud information is no longer reconstructed until the target position is reached, and the modified position is: x P ' = x p -d

[0020]

[0021] (III) Beneficial effects

[0022] Compared with the prior art, the present application provides an accurate model construction method of ultrasonic point cloud, which has the following beneficial effects:

[0023] 1. In the present application, by registration, the patient's clear tissue is obtained, so that the ultrasonic image and the CT image are in the same coordinate system, and the position of the cyst can be quickly found, which provides a basis for the subsequent accurate reconstruction.

[0024] 2. In the present application, by initially constructing the ultrasonic point cloud model, the motion path and scanning direction of the ultrasound can be determined, and the shape of the cyst can be preliminarily understood, which lays a foundation for accurate reconstruction.

[0025] 3. In the present application, by reconstructing the ultrasonic image twice, in the process of accurate reconstruction, the position compensation and contour compensation can be used to construct an accurate ultrasonic model, which avoids the problem that the partial volume effect will cause ultrasound artifacts, and sometimes the difference of a few millimeters will cause serious medical accidents. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 A flow structure diagram of an accurate model construction method of ultrasonic point cloud is provided for the present application;

[0027] Figure 2 An ultrasonic probe scanning cyst from left to right in a vertical direction is provided for the accurate model construction method of ultrasonic point cloud of the present application;

[0028] Figure 3 An ultrasonic image after scanning is provided for the accurate model construction method of ultrasonic point cloud of the present application;

[0029] Figure 4 An ultrasonic scanning diagram after position compensation is provided for the accurate model construction method of ultrasonic point cloud of the present application;

[0030] Figure 5 A schematic diagram of the placement position of an ultrasonic probe of an ultrasonic point cloud accurate model construction method proposed in the present application is shown in the figure.

[0031] Figure 6 A schematic diagram of a reconstructed accurate cyst model of an ultrasonic point cloud accurate model construction method proposed in the present application is shown in the figure. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0033] Reference Figures 1-6 An ultrasonic point cloud accurate model construction method, comprising the following steps:

[0034] Step 1: manually find the tumor position and point cloud;

[0035] Step 1-1: CT image point cloud, CT scanning is performed on the patient before the operation, and blood vessels and cysts are segmented and point cloud reconstruction is performed;

[0036] Step 1-2: ultrasonic image point cloud, a three-dimensional point cloud image is established according to the ultrasonic two-dimensional image and the mechanical arm pose information;

[0037] Step 2: initial reconstruction;

[0038] Step 2-1: by the ultrasonic probe installed on the mechanical arm, scanning along the fixed path in the fixed direction to obtain real-time ultrasonic information, specifically, on the basis of the known cyst position, the ultrasonic probe is rigidly connected with the mechanical arm, the mechanical arm is controlled to make the ultrasonic probe and the patient's skin form a fixed angle, and the cyst is scanned along the fixed route, as shown in Figure 2 The imaging and motion speed of the ultrasonic probe, the thickness of the probe and the motion direction are related, so the ultrasonic probe needs to be connected with the mechanical arm to ensure that the ultrasonic probe scans along the fixed direction and the fixed path;

[0039] Step 2-2: initial construction of ultrasonic point cloud model, the ultrasonic probe scans the cyst from left to right along the direction perpendicular to the place of the patient's skin, since the ultrasonic beam emitted by the probe has a certain thickness, that is, the obtained image is a superimposed image of the echo information within a certain thickness, as shown in Figure 3 A two-dimensional coordinate system is established, the thickness of the ultrasonic probe is d, then the two-dimensional position of the image set P scanned by the probe with a translation distance of L is:

[0040]

[0041] From formula (1), we can see that the actual cyst image I(A) and the ultrasound scan image I(B) will have two differences: at the x-axis position, due to the thickness of the ultrasound, the image will be formed in the middle of the ultrasound, which deviates from the actual position. In terms of image description, the scanned image is a collection of all images within the ultrasound scanning range, which will cause differences in image shape. Based on this, the I(B) points of the scanned image are cloudified.

[0042] Step 3: Precise reconstruction. Based on the initially reconstructed ultrasound point cloud model, the cyst is precisely reconstructed through reverse engineering.

[0043] Step 3-1: Position compensation, the compensation formula is as follows Where x P ' represents the x-axis coordinates of the accurately reconstructed point set P;

[0044] Step 3-2: Contour compensation, the ultrasonic probe is in Figure 5 When the ultrasonic probe first contacts the outermost edge of the contour, i.e., when it reaches position "1", the point cloud position is constructed according to formula (1). After passing position "1", the point cloud information is no longer reconstructed until position "2" is reached, at which point the position is corrected to: x P '=x p -d

[0045]

[0046] After the two compensations described above, an accurate cyst model can be reconstructed, such as... Figure 6 As shown.

[0047] Using the methods described above, ultrasound images are reconstructed twice in current ultrasound applications. During the precise reconstruction process, position compensation and contour compensation can be used to build an accurate ultrasound model, avoiding artifacts caused by partial volume effects. In puncture surgery, precise localization of cysts is required, and even slight errors can lead to serious medical accidents.

[0048] In the description of the text, it needs to be explained that the relationship terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment.

[0049] Although the embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for constructing an accurate model of ultrasonic point clouds, characterized in that, Includes the following steps: S1: CT image point cloudification. Before surgery, the patient undergoes a CT scan, and blood vessels and cysts are segmented and reconstructed using point cloud. S2: Ultrasonic image point cloudification. The ultrasonic probe mounted on the robotic arm continuously scans to acquire real-time ultrasonic video information, and then obtains the posture information corresponding to the ultrasonic video information and the ultrasonic probe posture information (x, y, z, γ, β, α). Then, the ultrasonic image is matched with the corresponding posture information. The identification of organs in the ultrasonic image and the display of the identification results are sampled. Based on the sampled organ contour points, three-dimensional coordinate transformation is performed to obtain the actual position of each contour point in the robotic arm coordinate system of the frame image. For sparse points, interpolation processing based on contour features is performed. Each transformed point is depicted in 3D space for visualization display. The coordinate system of this space is set according to the robot coordinate system. S3: Initial reconstruction. An ultrasound probe mounted on a robotic arm scans along a fixed path in a fixed direction to acquire real-time ultrasound information. Then, the ultrasound probe scans the cyst from left to right along a direction perpendicular to the patient's skin. Since the ultrasound beam emitted by the probe has a certain thickness, the obtained image is a superposition of spatial echo information within a certain thickness. In S3, the thickness of the ultrasound probe is d. As the probe moves a distance L, the two-dimensional position of the scanned image set P is: From formula (1), we can see that the actual cyst image I(A) and the ultrasound scan image I(B) will have two differences: at the x-axis position, due to the thickness of the ultrasound, the image will be formed in the middle of the ultrasound, which deviates from the actual position. In terms of image description, the scanned image is a collection of all images within the ultrasound scanning range, which will cause differences in image shape. Based on this, the I(B) points of the scanned image are cloudified. S4: Precise Reconstruction. Based on the initially reconstructed ultrasound point cloud model, precise reconstruction of the cyst is achieved through reverse engineering. Position and contour compensation are performed during precise reconstruction. The position compensation formula in S4 is as follows: Where x P ' To accurately reconstruct the x-axis coordinates of the point set P; contour compensation, when the ultrasonic probe is at a certain position, the overlap will cause the contour of the point cloud image to be deformed. Therefore, based on the initial reconstruction, when the ultrasonic probe first contacts the outermost point of the contour, that is, when it reaches position "1", the point cloud position is constructed according to formula (1). After passing position "1", the point cloud information is no longer reconstructed until it reaches position "2", and the position is corrected as follows: when the ultrasonic probe first contacts the outermost point of the contour, the point cloud position is constructed according to formula (1), and then the point cloud information is no longer reconstructed until the target position is reached, and the corrected position is: x P ' =x p -d 2. The method for constructing an accurate model of ultrasonic point clouds according to claim 1, characterized in that, In S1, the patient undergoes enhanced CT and ultrasound scans. After the scans, staff manually select easily identifiable blood vessels or cysts based on the patient's CT scan results, perform tissue segmentation on the CT images, mark the blood vessels and cysts, and reconstruct point cloud images.

3. The method for constructing an accurate model of ultrasonic point clouds according to claim 2, characterized in that, In step S2, image recognition is performed on each frame of the acquired image to extract the contour of the organ for which the model is to be built, and the contour is sampled to obtain contour sampling points.

4. The method for constructing an accurate model of ultrasonic point clouds according to claim 1, characterized in that, Specifically, in S2, the robot coordinate system, probe coordinate system, and image coordinate system are defined as {B}, {P}, and {I}, respectively, and the transformation matrix from {P} to {B} is: The transformation matrix from {I} to {P} is, 5. The method for constructing an accurate model of ultrasonic point clouds according to claim 1, characterized in that, In step S3, based on the known location of the cyst, the ultrasound probe is rigidly connected to the robotic arm, and the robotic arm is controlled so that the ultrasound probe is at a fixed angle to the patient's skin, scanning the cyst along a fixed route.

Images