Collinear constraint type free arm ultrasonic probe space calibration method and calibration system

Abstract

The invention discloses a collinear constraint type free arm ultrasonic probe space calibration method and system, and the method comprises the following steps: inputting calibration data; performing plane pose analysis to obtain a first target point set under an ultrasonic image coordinate system; performing spatial pose analysis to obtain a second target point set from the ultrasonic phantom position sensor coordinate system; outputting a calibration matrix; the calibration system comprises a calibration data acquisition module and a data format analysis module in the calibration data input process. The plane pose analysis process comprises an ultrasonic image segmentation module and a shape priori correction module; the spatial pose analysis process comprises a multi-physics field unification module; a matrix solving module is included in the calibration matrix output process. The automation degree of the calibration process is improved. Interference of noise points on calibration precision is reduced. According to the method, the precision and efficiency of ultrasonic probe space calibration are improved, and a foundation is laid for precise and efficient free arm three-dimensional ultrasonic imaging.

Description

Technical Field

[0001] The invention belongs to the technical fields of ultrasonic imaging, computer vision and rigid body kinematics, and in particular relates to a spatial calibration method and a calibration system for a collinearly constrained free-arm ultrasonic probe. Background Art

[0002] Medical ultrasound imaging technology is widely used in the fields of medical auxiliary diagnosis and auxiliary treatment due to its advantages such as simple use, low price, real-time imaging, percutaneous non-invasiveness, and no electromagnetic radiation. However, traditional two-dimensional ultrasound images can only provide tomographic images of tissues and organs but not three-dimensional images. Clinicians need to rely on experience to imagine in their brains. This process is not only time-consuming, labor-intensive and inefficient, but also subjective and prone to medical diagnostic errors. Due to the defects of traditional two-dimensional ultrasound imaging technology, three-dimensional ultrasound imaging technology has developed rapidly and been widely used in recent years.

[0003] At present, there are three main methods for acquiring three-dimensional ultrasound images: dedicated integrated ultrasound probes, two-dimensional array probes, and free-arm three-dimensional ultrasound technology. Although the first two can directly generate three-dimensional images, the data acquisition range is limited; free-arm three-dimensional ultrasound technology, by virtue of its unique flexibility, can record the spatial position and direction information of the probe in real time by installing a position sensor on the probe, thus achieving unlimited data acquisition range. Doctors can freely scan with the probe in hand, and then use reconstruction algorithms to fuse a series of two-dimensional ultrasound images into three-dimensional images, and intuitively display them through visualization technology, which greatly improves the accuracy and efficiency of medical diagnosis and treatment.

[0004] In the free-arm three-dimensional ultrasound technology, it is necessary to fix a position sensor on the ultrasound probe so that the tracking system can obtain the position and direction information of the ultrasound probe. In order for the positioning and tracking system to obtain the spatial position information of the two-dimensional ultrasound image, it is necessary to determine the spatial position conversion relationship between the ultrasound image coordinate system and the ultrasound probe coordinate system, and obtain the calibration matrix. This matrix can map the value of any pixel point in the two-dimensional ultrasound image to the ultrasound probe coordinate system, and then the tracking system can determine the position of any pixel in the ultrasound image in the world coordinate system. Therefore, the calibration of the ultrasound probe is a very critical step in realizing the free-arm three-dimensional ultrasound imaging system. The existing free-arm ultrasound probe calibration method mainly relies on manual point selection or complex iterative algorithms to achieve the target point extraction stage, which is not only inefficient, but also highly dependent on a single information source of pixel intensity during processing. This limitation makes it impossible to fully capture and consider the topological structure characteristics of the target object, which seriously restricts the accuracy and reliability of target point extraction.

[0005] In summary, the existing methods cannot achieve accurate and efficient free-arm ultrasound probe calibration process. Therefore, the present invention proposes a collinear constraint free-arm ultrasound probe space automatic calibration system and calibration method, which lays the foundation for accurate and efficient free-arm three-dimensional ultrasound imaging. Summary of the invention

[0006] The present invention is proposed to solve the problems existing in the prior art, and its purpose is to provide a spatial calibration method and a calibration system for a collinearly constrained free-arm ultrasonic probe.

[0007] The technical solution of the present invention is: a method for spatial calibration of a free-arm ultrasound probe with collinear constraint, comprising the following steps:

[0008] A. Input calibration data;

[0009] B. Performing plane pose analysis on the input calibration data to obtain the first target point set in the ultrasound image coordinate system;

[0010] C. Perform spatial posture analysis to obtain a second target point set derived from the ultrasonic phantom position sensor coordinate system;

[0011] D. Output calibration matrix.

[0012] Furthermore, step A performs calibration data input, and the specific process is as follows:

[0013] Firstly, establish a data transmission pipeline from binocular optical positioning equipment and ultrasonic imaging equipment to the image processing workstation;

[0014] Then, the image processing workstation simultaneously and synchronously collects the spatial posture data and plane image data received during the calibration data collection process, and classifies and stores them;

[0015] Finally, after the calibration data acquisition is completed, the image processing workstation classifies, reads and formats the stored calibration data.

[0016] Furthermore, step B performs plane pose analysis on the input calibration data to obtain a first target point set in the ultrasound image coordinate system. The specific process is as follows:

[0017] First, feature extraction is performed based on the convolutional neural network combined with the graph neural network for encoding, and then the convolutional neural network is decoded to automatically extract the locations of the calibrated points in the image.

[0018] Then, the automatically extracted results of the calibration points are subjected to contour extraction, contour counting, area sorting, centroid extraction, and grid division to obtain the centroid coordinates after screening and sorting;

[0019] Finally, according to the centroid coordinates after screening and sorting, the parallel constraint and the collinear constraint of each layer of N-line structure are implemented to obtain the first target point set in the ultrasound image coordinate system.

[0020] Furthermore, step C performs spatial posture analysis to obtain a second target point set derived from the ultrasonic phantom position sensor coordinate system. The specific process is as follows:

[0021] Firstly, the optical probe of the binocular optical positioning device is used to locate the marking points in the calibration phantom;

[0022] Then, the location of the N-line structure endpoints is realized by combining the positional relationship between the N-line structure endpoints and the marking points in the ultrasound phantom model coordinate system;

[0023] Finally, through the conversion relationship, the spatial pose of the calibration points in the ultrasound phantom position sensor coordinate system is mapped to the ultrasound probe coordinate system to obtain a second target point set originating from the ultrasound phantom position sensor coordinate system.

[0024] Furthermore, step D outputs the calibration matrix, and the specific process is as follows:

[0025] A calibration matrix is ​​obtained based on a first target point set in an ultrasound image coordinate system and a second target point set in an ultrasound phantom position sensor coordinate system.

[0026] A calibration system for a collinear constrained free-arm ultrasonic probe space calibration method, wherein the calibration data input process includes a calibration data acquisition module for acquiring data and a data format analysis module for analysis; the plane posture analysis process includes an ultrasonic image segmentation module for image segmentation and a shape priori correction module for correction; the space posture analysis process includes a multi-physical field unification module for physical field unification; and the calibration matrix output process includes a calibration matrix solution module for matrix solution.

[0027] Furthermore, the calibration data acquisition module includes a communication module, which establishes a data transmission pipeline from the binocular optical positioning device to the image processing workstation through serial port communication, and establishes a data transmission pipeline from the ultrasonic imaging device to the image processing workstation through the conversion of the image acquisition card.

[0028] Furthermore, the data format parsing module includes a reading module, which classifies and reads the spatial posture data and plane image data stored in the calibration data acquisition module, and transmits them to the subsequent ultrasound image segmentation module and multi-physical field unification module respectively.

[0029] Furthermore, the input data of the ultrasonic image segmentation module is the image data processed in the data format parsing module, and the output of the ultrasonic image segmentation module is the grayscale image data of the target point extracted from the original image data.

[0030] Furthermore, the multi-physics field unified module includes a multi-layer N-line phantom, which is composed of five layers of hollow structures. Each layer of the hollow structure includes 16 pairs of triangular prism through holes. The surface of the hollow structure also includes 16 marking points. By wiring in the triangular prism through holes, the N-line structure required in the calibration process can be built.

[0031] The beneficial effects of the present invention are as follows:

[0032] The present invention implements automatic extraction of target points in calibration images based on a method of combining a graph neural network with a convolutional neural network, reduces manual intervention, and improves the degree of automation of the calibration process.

[0033] The invention fits the multi-layer N-line structure in the collected image based on the least square method, adds parallel constraints and collinear constraints to the calibration features, and reduces the interference of noise points on the calibration accuracy.

[0034] The present invention improves the accuracy and efficiency of spatial calibration of ultrasound probes, laying a foundation for accurate and efficient free-arm three-dimensional ultrasound imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a module diagram of the calibration system in the present invention;

[0036] Figure 2 It is a module diagram of the calibration method in the present invention;

[0037] Figure 3 It is a coordinate system conversion relationship diagram in the present invention;

[0038] Figure 4 is a structural diagram of the ultrasonic calibration phantom in the present invention;

[0039] Figure 5 It is a network structure diagram of the calibration point extraction in the present invention;

[0040] Figure 6 It is a topological relationship correction diagram of the calibration points in the present invention. DETAILED DESCRIPTION

[0041] Hereinafter, the present invention will be described in detail with reference to the accompanying drawings and embodiments:

[0042] like Figures 1 to 6 As shown, a method for spatial calibration of a free-arm ultrasound probe with collinear constraint includes the following steps:

[0043] A. Input calibration data;

[0044] B. Performing plane pose analysis on the input calibration data to obtain the first target point set in the ultrasound image coordinate system;

[0045] C. Perform spatial posture analysis to obtain a second target point set derived from the ultrasonic phantom position sensor coordinate system;

[0046] D. Output calibration matrix.

[0047] Step A is to input calibration data. The specific process is as follows:

[0048] Firstly, establish a data transmission pipeline from binocular optical positioning equipment and ultrasonic imaging equipment to the image processing workstation;

[0049] Then, the image processing workstation simultaneously and synchronously collects the spatial posture data and plane image data received during the calibration data collection process, and classifies and stores them;

[0050] Finally, after the calibration data acquisition is completed, the image processing workstation classifies, reads and formats the stored calibration data.

[0051] Step B performs plane pose analysis on the input calibration data to obtain the first target point set in the ultrasound image coordinate system. The specific process is as follows:

[0052] First, feature extraction is performed based on the convolutional neural network combined with the graph neural network for encoding, and then the convolutional neural network is decoded to automatically extract the locations of the calibrated points in the image.

[0053] Then, the automatically extracted results of the calibration points are subjected to contour extraction, contour counting, area sorting, centroid extraction, and grid division to obtain the centroid coordinates after screening and sorting;

[0054] Finally, according to the centroid coordinates after screening and sorting, the parallel constraint and the collinear constraint of each layer of N-line structure are implemented to obtain the first target point set in the ultrasound image coordinate system.

[0055] Step C performs spatial posture analysis to obtain a second target point set derived from the ultrasonic phantom position sensor coordinate system. The specific process is as follows:

[0056] Firstly, the optical probe of the binocular optical positioning device is used to locate the marking points in the calibration phantom;

[0057] Then, the location of the N-line structure endpoints is realized by combining the positional relationship between the N-line structure endpoints and the marking points in the ultrasound phantom model coordinate system;

[0058] Finally, through the conversion relationship, the spatial pose of the calibration points in the ultrasound phantom position sensor coordinate system is mapped to the ultrasound probe coordinate system to obtain a second target point set originating from the ultrasound phantom position sensor coordinate system.

[0059] Step D outputs the calibration matrix. The specific process is as follows:

[0060] A calibration matrix is ​​obtained based on a first target point set in an ultrasound image coordinate system and a second target point set in an ultrasound phantom position sensor coordinate system.

[0061] A calibration system for a collinear constrained free-arm ultrasonic probe space calibration method, wherein the calibration data input process includes a calibration data acquisition module for acquiring data and a data format analysis module for analysis; the plane posture analysis process includes an ultrasonic image segmentation module for image segmentation and a shape priori correction module for correction; the space posture analysis process includes a multi-physical field unification module for physical field unification; and the calibration matrix output process includes a calibration matrix solution module for matrix solution.

[0062] The calibration data acquisition module includes a communication module, which establishes a data transmission pipeline from the binocular optical positioning device to the image processing workstation through serial port communication, and establishes a data transmission pipeline from the ultrasonic imaging device to the image processing workstation through the switching of the image acquisition card.

[0063] The data format parsing module includes a reading module, which classifies and reads the spatial posture data and plane image data stored in the calibration data acquisition module, and transmits them to the subsequent ultrasound image segmentation module and the multi-physical field unification module respectively.

[0064] The input data of the ultrasonic image segmentation module is the image data processed in the data format parsing module, and the output of the ultrasonic image segmentation module is the grayscale image data from which the target points in the original image data are extracted.

[0065] The multi-physics field unified module includes a multi-layer N-line phantom, which is composed of five layers of hollow structures. Each layer of the hollow structure includes 16 pairs of triangular prism through holes. The surface of the hollow structure also includes 16 marking points. By wiring in the triangular prism through holes, the N-line structure required in the calibration process is built.

[0066] Specifically, the calibration data acquisition module is used to synchronously acquire the spatial posture data in the binocular optical positioning device and the planar image data in the ultrasonic imaging device transmitted by the image acquisition card.

[0067] Specifically, the data format parsing module is used to convert spatial pose data storing information in quaternions into spatial pose data storing information in rotation matrices, and to convert image data storing information in pixel values ​​into graph data storing information in nodes and edges.

[0068] Specifically, the ultrasonic image segmentation module is used to implement feature extraction from the collected ultrasonic calibration image by jointly encoding with a convolutional neural network and a graph neural network, and then automatically extract the calibration point position in the image by decoding with a convolutional neural network.

[0069] Specifically, the shape prior correction module is used to extract the contour and mesh the area of ​​the calibration point from the neural network segmentation result, perform slope correction on the fitting straight line in each row, and perform coordinate correction on the calibration points around the fitting line.

[0070] Specifically, the multi-physics field unified module is used to transform the N-line structure endpoint coordinates in the ultrasound phantom model coordinate system into the ultrasound phantom position sensor coordinate system, and to transform the target point coordinates extracted from the ultrasound phantom position sensor coordinate system into the ultrasound probe coordinate system.

[0071] Specifically, the calibration matrix solving module defines an objective function based on a first target point set in an ultrasound image coordinate system and a second target point set in an ultrasound probe position sensor coordinate system, and solves the calibration matrix using a closed method.

[0072] Specifically, the calibration data acquisition module includes a storage module, which classifies and stores the acquired spatial posture data and plane image data, and is used for calibrating the data format parsing module in the system.

[0073] Specifically, the spatial posture data in the calibration data acquisition module comes from two aspects: one is the spatial posture of the calibration phantom tracked by the binocular optical positioning device, and the other is the spatial posture of the ultrasound probe tracked by the binocular optical positioning device. The reason why both can be tracked is that they are fixed to the reflective mark that can be recognized by the binocular optical positioning device.

[0074] Specifically, the data format parsing module includes graph data storing information in nodes and edges, wherein the edges used to connect the nodes have no direction, and the nodes have an eight-connected connected domain. The attribute of each node is a three-dimensional vector composed of the normalized result of the pixel intensity at the corresponding position in the original image data and the two-dimensional coordinates of the plane.

[0075] Specifically, when the upper-layer input data in the ultrasound image segmentation module passes through the current coding layer, it will simultaneously and separately pass through the graph convolution layer and the ordinary convolution layer for feature extraction, and then the extraction results of the two will be aggregated and feature compressed before being sent to the next layer of the neural network.

[0076] Specifically, when the shape prior correction module screens the contours, it will first count the contours. If the number of contours is less than 15, the image will be regarded as an abnormal image and the subsequent process will be terminated, and a new image will be selected for calibration. If the number of contours is greater than or equal to 15, the top 15 largest contours will be screened out according to the area enclosed by the contours.

[0077] More specifically, the shape prior correction module will solve the centroid of the 15 selected contours, and then divide the 15 centroid coordinates into 3 rows according to the size of the height coordinates, and then sort the 5 centroid coordinates in each row according to the size of the width coordinates, and finally form a centroid matrix with a grid distribution of 3 rows and 5 columns.

[0078] More specifically, the shape prior correction module solves the fitting line corresponding to each row of centroid point set according to the least square method, records the slope and offset, then averages the slopes of the three rows, and uses the slope average to correct the mean of the original fitting line to add parallel constraints between the three rows of N lines.

[0079] More specifically, the shape prior correction module projects the centroid points in each row onto the fitted straight line after the slope correction. During the projection process, the horizontal coordinates of the centroid points remain unchanged, and the vertical coordinates are translated to the corresponding fitted straight line, thereby implementing coordinate correction of the centroid point set in each image.

[0080] Specifically, in the multi-physics field unified module, 16 marking points are positioned by using an optical probe that is matched with a binocular optical positioning device, thereby realizing the positioning of the marking points in the coordinate system of the binocular optical positioning device and in the coordinate system of the ultrasonic phantom position sensor. Combined with the positional relationship between the endpoints of the N-line structure and the marking points in the coordinate system of the ultrasonic phantom model, the positioning of the endpoints of the N-line structure in the coordinate system of the ultrasonic phantom position sensor is obtained.

[0081] Specifically, in the multi-physics field unified module, based on the principle of similar triangles, the positioning of the marked points extracted from the collected calibration image in the coordinate system of the ultrasonic phantom position sensor is obtained, and then the spatial posture of the calibration points in the coordinate system of the ultrasonic phantom position sensor is mapped to the coordinate system of the ultrasonic probe through the coordinate system conversion relationship.

[0082] Specifically, the calibration matrix to be calculated in the calibration matrix solving module includes three translation parameters and three rotation parameters in three-dimensional space, and two scaling parameters in a two-dimensional plane.

[0083] Specifically, the input data of the calibration matrix solving module is the plane pose of the calibration data obtained by jointly analyzing the ultrasound image segmentation module and the shape prior correction module, and the spatial pose of the calibration data obtained by analyzing the multi-physical field unification module. The output result is the spatial position transformation matrix from the ultrasound image coordinate system to the ultrasound probe coordinate system to be obtained.

[0084] Embodiment 1

[0085] A method for spatial calibration of a free-arm ultrasound probe with collinear constraint, comprising the following steps:

[0086] A. Input calibration data;

[0087] B. Performing plane pose analysis on the input calibration data to obtain the first target point set in the ultrasound image coordinate system;

[0088] C. Perform spatial posture analysis to obtain a second target point set derived from the ultrasonic phantom position sensor coordinate system;

[0089] D. Output calibration matrix.

[0090] Step A is to input calibration data. The specific process is as follows:

[0091] Firstly, establish a data transmission pipeline from binocular optical positioning equipment and ultrasonic imaging equipment to the image processing workstation;

[0092] Then, the image processing workstation simultaneously and synchronously collects the spatial posture data and plane image data received during the calibration data collection process, and classifies and stores them;

[0093] Finally, after the calibration data acquisition is completed, the image processing workstation classifies, reads and formats the stored calibration data.

[0094] Step B performs plane pose analysis on the input calibration data to obtain the first target point set in the ultrasound image coordinate system. The specific process is as follows:

[0095] First, feature extraction is performed based on the convolutional neural network combined with the graph neural network for encoding, and then the convolutional neural network is decoded to automatically extract the locations of the calibrated points in the image.

[0096] Then, the automatically extracted results of the calibration points are subjected to contour extraction, contour counting, area sorting, centroid extraction, and grid division to obtain the centroid coordinates after screening and sorting;

[0097] Finally, according to the centroid coordinates after screening and sorting, the parallel constraint and the collinear constraint of each layer of N-line structure are implemented to obtain the first target point set in the ultrasound image coordinate system.

[0098] Step C performs spatial posture analysis to obtain a second target point set derived from the ultrasonic phantom position sensor coordinate system. The specific process is as follows:

[0099] Firstly, the optical probe of the binocular optical positioning device is used to locate the marking points in the calibration phantom;

[0100] Then, the location of the N-line structure endpoints is realized by combining the positional relationship between the N-line structure endpoints and the marking points in the ultrasound phantom model coordinate system;

[0101] Finally, through the conversion relationship, the spatial pose of the calibration points in the ultrasound phantom position sensor coordinate system is mapped to the ultrasound probe coordinate system to obtain a second target point set originating from the ultrasound phantom position sensor coordinate system.

[0102] Step D outputs the calibration matrix. The specific process is as follows:

[0103] A calibration matrix is ​​obtained based on a first target point set in an ultrasound image coordinate system and a second target point set in an ultrasound phantom position sensor coordinate system.

[0104] Specifically, step A performs calibration data input, as follows:

[0105] a1. Build as Figure 1 The calibration system shown includes a binocular optical positioning device, an ultrasonic imaging device, an image processing workstation, an image acquisition card, and a multi-layer N-line ultrasonic calibration phantom.

[0106] a2. The multi-layer N-line ultrasonic calibration phantom and the ultrasonic acquisition probe are both fixedly connected with reflective marks that can be recognized by the binocular optical positioning device, and the test reflective marks are within the recognition range of the binocular optical positioning device.

[0107] a3. Establish network communication from binocular optical positioning equipment to image processing workstation, and data transmission from ultrasonic imaging equipment to image processing workstation via image acquisition card.

[0108] a4. The handheld ultrasonic probe is used to scan the N-line structure in the calibration phantom in a water bath medium (alcohol concentration 8%, solution temperature 25°C), and the position data of the reflective mark and the image data of the ultrasonic scan are recorded at the same time.

[0109] a5. Convert the acquired data into a format that converts the pose data into a homogeneous transformation matrix and the image data into a graph structure consisting of nodes and edges.

[0110] More specifically, the transformation matrix includes a transformation matrix from the ultrasound phantom position sensor coordinate system to the binocular optical positioning device coordinate system And the transformation matrix from the binocular optical positioning device coordinate system to the ultrasound probe coordinate system

[0111] Specifically, step B performs plane pose analysis on the input calibration data to obtain a first target point set in the ultrasound image coordinate system. The specific process is as follows:

[0112] b1. Collect a batch of ultrasonic calibration images and manually mark the calibration points in them as training data. Figure 5 The neural network shown is trained by setting the loss function and optimization strategy so that it can acquire the ability to automatically extract the calibration points in the ultrasound image.

[0113] b2. In the calibration process, the graph structure data obtained through format conversion will first be sent to the trained neural network to implement feature extraction based on the joint encoding of convolutional neural network and graph neural network, and then the convolutional neural network will be decoded to automatically extract the location of the calibration points in the image.

[0114] b3. Perform contour extraction on the automatically extracted calibration point segmentation results, and then count the contours. If the number of contours is greater than or equal to 15, the top 15 largest contours are selected based on the area enclosed by the contours.

[0115] b4. Solve the centroids of the 15 screened contours, and then divide the 15 centroid coordinates into 3 rows according to the size of the height coordinates. Then sort the 5 centroid coordinates in each row according to the size of the width coordinates, and finally form a centroid matrix with a grid distribution of 3 rows and 5 columns.

[0116] b5. Solve the fitting line corresponding to each row of centroid point set using the least squares method, record the slope and bias. Then calculate the average of the slopes of the three rows, and use the slope average to correct the mean of the original fitting line to add parallel constraints between the three rows of N lines.

[0117] b6. Project the centroid points in each row onto the slope-corrected fitting line. During the projection process, the horizontal coordinate of the centroid points remains unchanged, and the vertical coordinate is translated to the corresponding fitting line. This corrects the coordinates of the centroid point set in the image, and finally obtains the first target point set in the ultrasound image coordinate system required for solving the calibration matrix. US p.

[0118] Specifically, step C performs spatial posture analysis to obtain a second target point set derived from the ultrasonic phantom position sensor coordinate system. The specific process is as follows:

[0119] c1. Use the optical probe to select 16 marking points on the surface of the multi-layer N-line phantom, denoted as World p mark , and then according to the conversion relationship The coordinates of the marker points are converted to the coordinate system of the ultrasonic phantom position sensor, denoted as Phantom p mark , the conversion process is as follows:

[0120]

[0121] c2. Based on the coordinates of the 16 markers in the ultrasound phantom position sensor coordinate system Phantom p mark and the coordinates in the ultrasound phantom model coordinate system Model p mark , the transformation relationship between the two coordinate systems can be calculated, denoted as The objective function based on the least square error criterion used in the solution process is as follows:

[0122]

[0123] Where i represents each of the 16 markers.

[0124] c3. Based on the conversion relationship Combined with the coordinates of the N-line structure endpoints in the ultrasound phantom model coordinate system Model p end , to realize the positioning of the endpoints of the N-line structure in the coordinate system of the ultrasonic phantom position sensor, denoted as Phantom p end , the positioning process is as follows:

[0125]

[0126] c4. Based on the intersection of the ultrasound imaging surface and the multi-layer N-line structure, it is constructed as shown in the attached figure. Figure 4 The similar triangle structure shown can realize the positioning of the calibration point in the coordinate system of the ultrasound phantom position sensor Phantom p.

[0127] c5. Based on conversion relationship and The coordinates of the calibration points in the ultrasonic phantom position sensor coordinate system can be mapped to the ultrasonic probe coordinate system, and finally the second target point set from the ultrasonic phantom position sensor coordinate system required for solving the calibration matrix can be obtained by analysis.

[0128] Specifically, step D outputs the calibration matrix, and the specific process is as follows:

[0129] d1. According to Figure 3 The coordinate system transformation relationship shown in the figure is the calibration point set in the ultrasound image coordinate system. US p and the calibration point set in the ultrasonic phantom position sensor coordinate system Phantom The mapping relationship between p can be expressed as:

[0130]

[0131] In the formula, This is the target matrix for ultrasonic calibration.

[0132] d2. Calibration point set based on the analytically obtained ultrasound image coordinate system US p and the calibration point set in the ultrasonic phantom position sensor coordinate system Phantom p, is solved using the following objective function based on the least squares error criterion and a closed-form method:

[0133]

[0134] Where i represents each element in the calibration point set obtained by analysis.

[0135] d3. After the calibration matrix is ​​solved, any pixel in the two-dimensional ultrasound image is mapped to the ultrasound probe coordinate system, and then the position of any pixel in the ultrasound image in the binocular optical positioning device coordinate system can be determined:

[0136]

[0137] Specifically, the operation process of the present invention is as follows:

[0138] First, open the calibration program in the image processing workstation and establish a data transmission pipeline between the hardware in the calibration system.

[0139] Then, the position data in the binocular optical positioning device and the image data in the ultrasonic imaging device at the same time are synchronously acquired.

[0140] Then, the plane pose of the image data is analyzed into the ultrasound image coordinate system, and the spatial pose of the pose data is analyzed into the ultrasound probe coordinate system.

[0141] Finally, based on the two calibration point sets obtained analytically, the calibration matrix is ​​solved using the least square standard deviation in the ultrasound probe coordinate system.

[0142] The above description of the present invention is merely illustrative and not restrictive, so the embodiments of the present invention are not limited to the above specific embodiments. If a person skilled in the art is inspired by the above, without departing from the scope of protection of the present invention and the claims, other changes or modifications are made, which belong to the protection scope of the present invention.

Claims

1. A spatial calibration method for a free-arm ultrasound probe with collinear constraints. Features: The following steps are involved: A. Input calibration data; B. Performing plane pose analysis on the input calibration data to obtain the first target point set in the ultrasound image coordinate system; C. Perform spatial posture analysis to obtain a second target point set derived from the ultrasonic phantom position sensor coordinate system; D. Output calibration matrix.

2. According to the method for spatial calibration of a collinearly constrained free-arm ultrasound probe according to claim 1, Features: Step A is to input calibration data. The specific process is as follows: Firstly, establish a data transmission pipeline from binocular optical positioning equipment and ultrasonic imaging equipment to the image processing workstation; Then, the image processing workstation simultaneously and synchronously collects the spatial posture data and plane image data received during the calibration data collection process, and classifies and stores them; Finally, after the calibration data acquisition is completed, the image processing workstation classifies, reads and formats the stored calibration data.

3. According to the collinear constraint free-arm ultrasound probe spatial calibration method of claim 1, Features: Step B performs plane pose analysis on the input calibration data to obtain the first target point set in the ultrasound image coordinate system. The specific process is as follows: First, feature extraction is performed based on the convolutional neural network combined with the graph neural network for encoding, and then the convolutional neural network is decoded to automatically extract the locations of the calibrated points in the image. Then, the automatically extracted results of the calibration points are subjected to contour extraction, contour counting, area sorting, centroid extraction, and grid division to obtain the centroid coordinates after screening and sorting; Finally, according to the centroid coordinates after screening and sorting, the parallel constraint and the collinear constraint of each layer of N-line structure are implemented to obtain the first target point set in the ultrasound image coordinate system.

4. According to the method for spatial calibration of a collinearly constrained free-arm ultrasound probe according to claim 1, Features: Step C performs spatial posture analysis to obtain a second target point set derived from the ultrasonic phantom position sensor coordinate system. The specific process is as follows: Firstly, the optical probe of the binocular optical positioning device is used to locate the marking points in the calibration phantom; Then, the location of the N-line structure endpoints is realized by combining the positional relationship between the N-line structure endpoints and the marking points in the ultrasound phantom model coordinate system; Finally, through the conversion relationship, the spatial pose of the calibration points in the ultrasound phantom position sensor coordinate system is mapped to the ultrasound probe coordinate system to obtain a second target point set originating from the ultrasound phantom position sensor coordinate system.

5. According to the method for spatial calibration of a collinearly constrained free-arm ultrasound probe according to claim 1, Features: Step D outputs the calibration matrix. The specific process is as follows: A calibration matrix is ​​obtained based on a first target point set in an ultrasound image coordinate system and a second target point set in an ultrasound phantom position sensor coordinate system.

6. A calibration system for a free-arm ultrasound probe spatial calibration method with collinear constraints, Features: The calibration data input process includes a calibration data acquisition module for acquiring data and a data format analysis module for analysis; the plane posture analysis process includes an ultrasound image segmentation module for image segmentation and a shape prior correction module for correction; the space posture analysis process includes a multi-physical field unification module for physical field unification; the calibration matrix output process includes a calibration matrix solution module for matrix solution.

7. The calibration system of the free-arm ultrasound probe spatial calibration method with collinear constraint according to claim 6, Features: The calibration data acquisition module includes a communication module, which establishes a data transmission pipeline from the binocular optical positioning device to the image processing workstation through serial port communication, and establishes a data transmission pipeline from the ultrasonic imaging device to the image processing workstation through the switching of the image acquisition card.

8. The calibration system of the free-arm ultrasound probe spatial calibration method with collinear constraint according to claim 6, Features: The data format parsing module includes a reading module, which classifies and reads the spatial posture data and plane image data stored in the calibration data acquisition module, and transmits them to the subsequent ultrasound image segmentation module and the multi-physical field unification module respectively.

9. The calibration system of the method for calibrating a free-arm ultrasonic probe with collinear constraint according to claim 6, Features: The input data of the ultrasonic image segmentation module is the image data processed in the data format parsing module, and the output of the ultrasonic image segmentation module is the grayscale image data from which the target points in the original image data are extracted.

10. The calibration system of the free-arm ultrasound probe spatial calibration method with collinear constraint according to claim 6, Features: The multi-physics field unified module includes a multi-layer N-line phantom, which is composed of five layers of hollow structures. Each layer of the hollow structure includes 16 pairs of triangular prism through holes. The surface of the hollow structure also includes 16 marking points. By wiring in the triangular prism through holes, the N-line structure required in the calibration process is built.

Images