Calibration method and calibration system of ultrasonic probe and electronic equipment

Abstract

The embodiment of the invention is suitable for the technical field of medical imaging and image calibration, and provides an ultrasonic probe calibration method and system and electronic equipment, and the method comprises the steps: receiving an ultrasonic probe pose sequence and a calibration model pose sequence; receiving an ultrasonic image sequence; performing mark point identification and straight line identification on the ultrasonic image sequence to determine a first noise image and a second noise image in the ultrasonic image sequence; after the second noise image is deleted from the ultrasonic image sequence, time calibration is carried out on the ultrasonic probe, and a rate difference value between an ultrasonic probe displacement signal and an ultrasonic image displacement signal is obtained; and after deleting the first noise image from the ultrasonic image sequence, performing spatial pose calibration on the ultrasonic probe based on the rate difference value to obtain a spatial pose calibration matrix of the ultrasonic probe. By adopting the method, the calibration precision of the ultrasonic probe and the stability of the calibration result can be improved.

Description

Technical Field

[0001] The embodiments of the present application belong to the technical fields of medical imaging technology and image calibration technology, and particularly relate to a calibration method, a calibration system and an electronic device for an ultrasonic probe. Background Art

[0002] With the development of medical imaging, image-guided technology has become increasingly important in surgical operations. Real-time intraoperative imaging and accurate positioning are the keys to successful surgery. As a safe and non-invasive imaging technology, ultrasonic imaging has been widely used in many surgical fields, especially in orthopedics, neurosurgery, oncology surgery, cardiac surgery and other fields. Imaging technology based on robot-assisted navigation has significant advantages in surgical operations, especially in minimally invasive surgeries. Compared with traditional surgeries, the application of this technology can effectively improve the success rate and safety of surgeries. Commonly used robot-assisted navigation imaging technologies include X-ray imaging, CT imaging, ultrasonic imaging, etc.

[0003] Radiographic imaging technologies such as X-ray imaging and CT imaging obtain static image data. The static image data can be used to register the preoperative image and the intraoperative image of the patient, or to register the real position of the patient and the intraoperative image, through rigid transformation and non-rigid transformation. This registration method has high accuracy and good stability, and is most widely used in surgical navigation. However, the application of radiographic imaging technology will cause more radiation exposure to patients and doctors.

[0004] Ultrasonic imaging technology does not cause radiation to patients and doctors. In intraoperative ultrasonic imaging, usually a doctor holds an ultrasonic probe and scans the area or part of the patient that needs to be operated on, to obtain a dynamic two-dimensional (2D) ultrasonic image sequence. According to the obtained 2D image sequence, it is registered with the preoperative image, or with the real position of the patient during the operation, to achieve the purpose of intraoperative ultrasonic guidance. The accuracy of this registration method depends on the calibration of the ultrasonic probe. Since the calibration process of the ultrasonic probe is cumbersome and complex, and the stability of the calibration result is poor, ultrasonic imaging technology is less used in surgical navigation. Summary of the Invention

[0005] In view of this, the embodiments of the present application provide a calibration method, a calibration system and an electronic device for an ultrasonic probe, which are used together to calibrate the ultrasonic probe by combining time calibration and space calibration, so as to improve the calibration accuracy of the ultrasonic probe and the stability of the calibration result.

[0006] The first aspect of the embodiments of the present application provides a calibration method for an ultrasonic probe, including: The receiving optical navigation device receives the ultrasonic probe pose sequence and the calibration model pose sequence obtained by identifying the first tracer and the second tracer. The first tracer is installed on the ultrasonic probe, and the second tracer is installed on the calibration model. The calibration model is a multi-layer N-line model with acoustic wave reflection material at the bottom; Receive the ultrasonic image sequence obtained by the ultrasonic device through collecting ultrasonic images. The ultrasonic images are obtained by collecting during the process of placing the ultrasonic probe on the calibration model and sliding it; By performing marker point recognition and straight line recognition on the ultrasonic image sequence, determine the first noise image and the second noise image in the ultrasonic image sequence. The first noise image is an image in the ultrasonic image sequence that does not meet the marker point recognition requirements, and the second noise image is an image in the ultrasonic image sequence that does not meet the straight line recognition requirements; After deleting the second noise image from the ultrasonic image sequence, perform time calibration on the ultrasonic probe to obtain the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal; After deleting the first noise image from the ultrasonic image sequence, perform spatial pose calibration on the ultrasonic probe based on the rate difference to obtain the spatial pose calibration matrix of the ultrasonic probe.

[0007] The second aspect of the embodiments of the present application provides a calibration device for an ultrasonic probe, including: A pose sequence receiving module, configured to receive the ultrasonic probe pose sequence and the calibration model pose sequence obtained by the optical navigation device through identifying the first tracer and the second tracer. The first tracer is installed on the ultrasonic probe, and the second tracer is installed on the calibration model. The calibration model is a multi-layer N-line model with acoustic wave reflection material at the bottom; An image sequence receiving module, configured to receive the ultrasonic image sequence obtained by the ultrasonic device through collecting ultrasonic images. The ultrasonic images are obtained by collecting during the process of placing the ultrasonic probe on the calibration model and sliding it; A noise image recognition module, configured to determine the first noise image and the second noise image in the ultrasonic image sequence by performing marker point recognition and straight line recognition on the ultrasonic image sequence. The first noise image is an image in the ultrasonic image sequence that does not meet the marker point recognition requirements, and the second noise image is an image in the ultrasonic image sequence that does not meet the straight line recognition requirements; A time calibration module, configured to perform time calibration on the ultrasonic probe after deleting the second noise image from the ultrasonic image sequence to obtain the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal; A spatial calibration module, configured to perform spatial pose calibration on the ultrasonic probe based on the rate difference after deleting the first noise image from the ultrasonic image sequence, so as to obtain a spatial pose calibration matrix of the ultrasonic probe.

[0008] In a third aspect of the embodiments of the present application, there is provided a calibration system for an ultrasonic probe, including a calibration model, an electronic device, an optical navigation device and an ultrasonic device connected to the electronic device. The calibration model is a multi-layer N-line model with a sound wave reflection material at the bottom. The ultrasonic device includes an ultrasonic probe, and a first tracer and a second tracer are respectively installed on the ultrasonic probe and the calibration model. Wherein: The optical navigation device is configured to obtain an ultrasonic probe pose sequence and a calibration model pose sequence by identifying the first tracer and the second tracer; The ultrasonic device is configured to collect ultrasonic images during the process of placing the ultrasonic probe on the calibration model and sliding it; The electronic device is configured to implement the method described in the first aspect above.

[0009] In a fourth aspect of the embodiments of the present application, there is provided an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device is enabled to implement the method described in the first aspect above.

[0010] In a fifth aspect of the embodiments of the present application, there is provided a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a computer, the method described in the first aspect above is implemented.

[0011] In a sixth aspect of the embodiments of the present application, there is provided a computer program product, including a computer program. When the computer program runs, the method described in the first aspect above is executed.

[0012] Compared with the prior art, the embodiments of the present application have the following beneficial effects: In the embodiments of the present application, the error existing in identifying the marker points from the ultrasonic images is reduced through image recognition; by screening and discarding the noise data in the collected data, the accuracy of the subsequent processing results is ensured; by jointly performing time calibration and spatial pose calibration on the ultrasonic probe, the problem of inconsistent data transmission rates between different ultrasonic devices and optical navigation devices is solved, and it can be ensured that the ultrasonic image sequence, the ultrasonic probe pose sequence and the calibration model pose sequence required for calibration correspond one by one. Applying the calibration method provided by the embodiments of the present application can improve the calibration accuracy of the ultrasonic probe and the stability of the calibration results. Description of the Drawings

[0013] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for use in the embodiments or the description of the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0014] Figure 1 It is a schematic diagram of a calibration method for an ultrasonic probe provided by an embodiment of the present application; Figure 2 It is a schematic diagram of a calibration process for an ultrasonic probe provided by an embodiment of the present application; Figure 3 It is a schematic diagram of a calibration system for an ultrasonic probe provided by an embodiment of the present application; Figure 4 It is a schematic diagram of a calibration device for an ultrasonic probe provided by an embodiment of the present application; Figure 5 It is a schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0015] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are presented to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, the detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0016] As introduced in the background art, during intraoperative ultrasound imaging, the doctor mainly holds the ultrasonic probe and scans the area or part of the patient that needs to be operated on, and then uses the obtained dynamic 2D ultrasound image sequence to register with the preoperative image or the real position of the patient during the operation, so as to achieve the purpose of intraoperative ultrasound guidance. In the related art, the calibration of the ultrasonic probe is only for spatial pose calibration, and imaging errors such as ghosting, missing images, and afterimages will inevitably occur during the scanning process. In addition, problems such as the inconsistency of image processing and transmission rates between ultrasonic devices and the large difference in the output rates of optical navigation devices will also affect the calibration process. These problems may all lead to a large error in the transformation matrix that needs to be calculated during the calibration process, and thus lead to a larger calibration error.

[0017] In view of the above problems, the embodiments of the present application provide a calibration method, device, and electronic device for an ultrasonic probe, which can be used together with time calibration and spatial calibration to calibrate the ultrasonic probe, so as to improve the calibration accuracy of the ultrasonic probe and the stability of the calibration result.

[0018] The technical solution of the present application will be described below through specific embodiments.

[0019] Referring to Figure 1 , a schematic diagram of a calibration method for an ultrasonic probe provided by an embodiment of the present application is shown, which may specifically include the following steps: S101. Receive the ultrasonic probe pose sequence and the calibration model pose sequence obtained by the optical navigation device by identifying the first tracer and the second tracer.

[0020] It should be noted that the embodiments of the present application can be applied to an electronic device, that is, the execution subject of this method can be an electronic device. By executing the various steps of the method provided by the embodiments of the present application, the electronic device can be used in combination with time calibration and space calibration to jointly calibrate the ultrasonic probe and obtain a calibration result with high accuracy and high stability. The above-mentioned electronic device can be a desktop computer, a cloud server, etc. The embodiments of the present application do not limit the specific type of the electronic device.

[0021] During the calibration process of the ultrasonic probe, a calibration model can be used. The calibration model applied in the embodiments of the present application can be a multi-layer N-line model with a sound wave reflection material at the bottom. In each layer of the calibration model, a group of parallel line segments can be formed by pulling wires. By pulling wires to connect the two endpoints at different ends of the parallel line segments again, a graph similar to the letter "N" can be formed, which is the N-line. The ordinary N-line model is mostly a 3-layer model without a bottom surface. In the embodiments of the present application, the bottom surface of the N-line model is made of a sound wave reflection material, which can reflect sound waves through the sound wave reflection material on the bottom surface during the calibration process of the ultrasonic probe, helping to obtain clearer imaging in the ultrasonic image. The number of layers of the N-line model in the embodiments of the present application can be determined according to the actual needs of the area to be scanned by the ultrasonic device. Usually, the number of layers of the N-line model can be positively correlated with the depth of the area to be scanned. For example, for an area to be scanned with a depth of about 10 cm, a 3-5 layer N-line model can be used to calibrate the ultrasonic probe. For an area to be scanned with a depth of 15 cm, 20 cm or deeper, an 8-layer or more N-line model can be used to calibrate the ultrasonic probe.

[0022] In the embodiments of the present application, tracers (markers) can be installed on both the ultrasonic probe and the calibration model. For example, the first tracer and the second tracer are used to distinguish the above two tracers. The first tracer can be installed on the ultrasonic probe, and the second tracer can be installed on the calibration model. In this way, during the actual calibration process, the optical navigation device can obtain the ultrasonic probe pose sequence and the calibration model pose sequence by identifying the two tracers.

[0023] Both the ultrasonic probe pose sequence and the calibration model pose sequence include multiple pose matrices, and each pose matrix can be decomposed into an attitude matrix and a position matrix. Accordingly, combining multiple attitude matrices together can be called an attitude sequence, and combining multiple position matrices together can be called a position sequence.

[0024] In a specific implementation, before starting the electronic device and the optical navigation device, the optical navigation device can be placed within the optimal acquisition range through settings. In this way, it helps to more accurately identify each tracer and obtain a more accurate and complete pose sequence.

[0025] The ultrasonic probe pose sequence and the calibration model pose sequence obtained by the optical navigation device by identifying the first tracer and the second tracer can be transmitted to the electronic device, and the electronic device can perform further processing in combination with the ultrasonic image sequence collected by the ultrasonic device.

[0026] S102. Receive the ultrasonic image sequence obtained by the ultrasonic device by collecting ultrasonic images.

[0027] During the process of calibrating the ultrasonic probe, the ultrasonic probe can be placed on the calibration model and slid. During the sliding of the ultrasonic probe, the ultrasonic device can collect ultrasonic images. Each collected ultrasonic image is a two-dimensional image. The continuously collected ultrasonic images can form an ultrasonic image sequence. The ultrasonic image sequence can be transmitted from the ultrasonic device to the electronic device.

[0028] In the embodiment of the present application, the electronic device is connected to the optical navigation device and the ultrasonic device. When calibrating the ultrasonic probe, the electronic device can enable the multi-threaded acquisition function to simultaneously receive the ultrasonic probe pose sequence and the calibration model pose sequence recognized by the optical navigation device, and receive the ultrasonic image sequence collected by the ultrasonic device. The pose matrices in the ultrasonic probe pose sequence, the pose matrices in the calibration model pose sequence, and the ultrasonic images in the ultrasonic image sequence have a corresponding relationship. Exemplarily, when the ultrasonic probe slides on the calibration model, the ultrasonic device will collect an ultrasonic image. At this time, the optical navigation device can obtain the ultrasonic probe pose matrix and the calibration model pose matrix by identifying the tracers on the ultrasonic probe and the calibration model. The ultrasonic image, the ultrasonic probe pose matrix, and the calibration model pose matrix at this moment have a corresponding relationship.

[0029] S103. Determine the first noise image and the second noise image in the ultrasonic image sequence by performing marker point recognition and line recognition on the ultrasonic image sequence.

[0030] After receiving data such as the ultrasonic probe pose sequence, the calibration model pose sequence, and the ultrasonic image pose sequence, the electronic device can process the received data to identify ultrasonic images with imaging errors and ultrasonic images with inconsistent pose sequences. In the embodiments of the present application, the above images can be referred to as noise images.

[0031] In the embodiments of the present application, image processing can be performed on the ultrasonic images. By identifying the marker points and straight lines in each ultrasonic image included therein, and determining the images that do not meet the corresponding identification requirements as noise images. Specifically, the images in the ultrasonic image sequence that do not meet the marker point identification requirements can be marked as the first noise images, and the images in the sequence that do not meet the straight line identification requirements can be marked as the second noise images. The above first noise images and second noise images will be discarded in the subsequent processing. For example, when performing time calibration, the second noise images can be discarded; after completing time calibration and performing spatial pose calibration, the first noise images can be discarded. When the first noise images and the second noise images are discarded, their corresponding ultrasonic probe pose matrices and calibration model pose matrices will also be discarded from the ultrasonic probe pose sequence or the calibration model pose sequence.

[0032] In the embodiments of the present application, the first noise images in the ultrasonic images that do not meet the marker point identification requirements can be determined through marker point identification. The above marker point identification can refer to identifying the two-dimensional (2D) marker points in the ultrasonic images. The 2D marker points in the ultrasonic images are derived from the line segments in the calibration model, that is, the line segments in the shape of the letter "N" introduced above. That is, the marker points in the ultrasonic image sequence can be used to represent the line segments in the calibration model. Since each layer of the calibration model contains a figure in the shape of the letter "N", correspondingly, each layer of the calibration model includes 3 line segments. Therefore, the total number of line segments in the calibration model depends on the number of layers of the model. For example, assuming that the calibration model is a 6-layer N-line model, since each layer includes 3 line segments, the 6-layer N-line model has a total of 18 line segments.

[0033] According to the properties of ultrasonic imaging, each 2D marker point in the ultrasonic image will appear as an irregular small area in the image. Therefore, by performing image processing and analysis on each collected ultrasonic image, the 2D marker points included therein can be identified.

[0034] In a possible implementation manner of the embodiments of the present application, when identifying the marker points in the ultrasonic image sequence to determine the first noise images in the sequence, based on the number of line segments in the calibration model, connected component analysis can be performed on each ultrasonic image in the ultrasonic image sequence to obtain multiple collinear connected component groups, and based on the multiple collinear connected component groups, multiple candidate marker points can be determined.

[0035] In the embodiments of the present application, the electronic device may first preprocess each ultrasonic image, such as performing histogram equalization, threshold segmentation, etc. on the ultrasonic image. Then, the electronic device performs connected component analysis on the image, identifies multiple connected components included in each ultrasonic image in the ultrasonic image sequence, and filters out multiple candidate connected components with the same number according to the number of line segments in the calibration model.

[0036] Specifically, after the electronic device obtains multiple connected components included in each image by performing connected component analysis on the ultrasonic image, it can identify the region of interest (ROI) according to the characteristics of the connected components. By selecting the ROI region, the algorithm speed can be improved. Then, by calculating the area and centroid of each connected component, the average area can be obtained. According to the number of line segments in the calibration model, the identified connected components can be filtered to obtain candidate connected components. For example, assume that the calibration model is a 6-layer N-line model with a total of 18 line segments. Therefore, the connected components identified in the ultrasonic image should theoretically include 18. Assume that 24 connected components are actually identified. The 24 connected components can be sorted according to the area size of each connected component. Taking the average area as the benchmark, 9 connected components are selected from the left and right respectively, and the selected 18 connected components are used as candidate connected components. That is, the electronic device can calculate the area of each identified connected component, sort the connected components according to the area size, and select multiple connected components as candidate connected components in the order of area size taking the average area as the benchmark, so that the number of selected candidate connected components is equal to the number of line segments in the calibration model.

[0037] Based on filtering out multiple candidate connected components, the electronic device can perform collinearity analysis on the multiple candidate connected components to obtain multiple groups of collinear connected components. The number of the above multiple groups of collinear connected components is the same as the number of layers of the calibration model. For example, when calibrating using a 6-layer N-line model, the number of groups of collinear connected components filtered out should be 6. In this way, the electronic device can respectively perform linear fitting on the centroids of the connected components in each group of collinear connected components, and determine the obtained fitting points as candidate marker points.

[0038] Specifically, the electronic device can perform collinearity analysis on each centroid with the horizontal direction as the search direction according to the centroid of each candidate connected component to obtain collinear connected components, which form a group of collinear connected components. Each group of collinear connected components includes 3 collinear connected components. By performing linear fitting on the centroids of these 3 collinear connected components, the obtained fitting points can be used as candidate 2D marker points.

[0039] After completing the recognition of 2D marker points, the electronic device can verify multiple candidate marker points. By verifying multiple candidate marker points, the electronic device can identify the ultrasound images corresponding to the candidate marker points that do not meet the verification requirements as the first noise images. The above verification of multiple candidate marker points can be called the first verification. That is, by performing the first verification on multiple candidate marker points, the ultrasound images corresponding to the candidate marker points that do not meet the first verification requirements are identified as the first noise images.

[0040] In the embodiment of the present application, the first verification may include a parallelism verification and a distance verification. The electronic device can first perform a parallelism verification on the candidate marker points. Specifically, the electronic device can calculate the slopes of the lines fitted by multiple candidate marker points in each collinear connected domain group respectively. For example, the line fitted by multiple candidate marker points can be called the first line, and correspondingly, the slope of this line can be called the first slope. If the above first slope is within the preset first slope range, the electronic device can determine that the ultrasound images corresponding to multiple candidate marker points in the collinear connected domain group pass the parallelism verification.

[0041] For the candidate marker points that pass the parallelism verification, the electronic device can perform a distance verification on them. Specifically, the electronic device can calculate the distances between each collinear connected domain group respectively. If this distance is within the preset distance range, it can be determined that the ultrasound images corresponding to multiple candidate marker points in the collinear connected domain group pass the distance verification.

[0042] The electronic device can identify the ultrasound images that do not pass the parallelism verification or the distance verification as the first noise images. After completing the subsequent time calibration and when spatial pose calibration is required, the ultrasound images identified as the first noise images and their corresponding ultrasound probe pose matrices and calibration model pose matrices will be discarded from the image sequence and the pose sequence.

[0043] After completing the recognition of the first noise images, the electronic device can determine the second noise images in the ultrasound image sequence through line recognition. The second noise images are also the ultrasound images that do not meet the above line recognition requirements.

[0044] When performing straight line recognition on ultrasonic images, the ultrasonic images can also be preprocessed first. For example, preprocessing such as image denoising and contrast enhancement can be performed on the ultrasonic images. Then, each ultrasonic image in the ultrasonic image sequence can be cropped horizontally to obtain multiple cropped regions of each ultrasonic image. In this way, each ultrasonic image is cropped into multiple strip-shaped regions. On this basis, the electronic device can determine the region maximum value points corresponding to the pixel maximum values in each cropped region of each ultrasonic image and record the coordinates of all region maximum value points. For an ultrasonic image, the number of region maximum value points it includes is equal to the number of regions into which the ultrasonic image is cropped. For example, if an ultrasonic image is cropped into 8 regions, 8 region maximum value points can be obtained by determining the pixel maximum values in each cropped region.

[0045] Then, the electronic device can verify the corresponding ultrasonic image based on the region maximum value points in each cropped region. To distinguish it from the foregoing first verification, the verification at this time can be called the second verification. Ultrasonic images that do not meet the requirements of the second verification can be identified as second noise images.

[0046] Specifically, the electronic device can calculate the slope of the straight line obtained by fitting based on the region maximum value points in each cropped region. For example, this straight line can be called the second straight line, and correspondingly, its slope can be the second slope. If the above second slope is within the preset second slope range, it can be determined that the corresponding ultrasonic image passes the second verification. The electronic device can identify ultrasonic images that do not pass the second verification as second noise images.

[0047] In the embodiments of the present application, for ultrasonic images that pass the second verification, that is, ultrasonic images that are not identified as second noise images, the electronic device can determine the intersection points of the second straight line obtained by fitting in each ultrasonic image with the vertical midline of the current ultrasonic image to obtain an intersection point sequence corresponding to multiple ultrasonic images.

[0048] Specifically, a straight line on the ultrasonic image can be obtained by fitting based on the region maximum value points in each cropped region, that is, the foregoing second straight line. At the same time, the electronic device can draw a vertical line, that is, the vertical midline, through the midpoint of the image on this ultrasonic image. The vertical midline and the second straight line obtained by fitting will intersect to form an intersection point. The same processing is performed on each ultrasonic image that passes the second verification, and an intersection point can be obtained on each ultrasonic image. These intersection points can be combined to form an intersection point sequence.

[0049] The electronic device can perform normalization processing on each intersection point in the intersection point sequence based on the vertical direction of the image to obtain an ultrasonic image displacement signal. This displacement signal can represent the actual displacement situation of the ultrasonic images collected during the calibration process.

[0050] In the image coordinate system, the vertical direction of the image is the y-axis direction. Therefore, the above normalization process for each intersection point based on the vertical direction of the image can refer to normalizing the x-axis coordinate value in the coordinates of each intersection point, and the range of the normalization process is (1, 0).

[0051] S104. After deleting the second noise image from the ultrasonic image sequence, perform time calibration on the ultrasonic probe to obtain the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal.

[0052] After completing the processing of the foregoing S101 - S103 steps, the electronic device can perform time calibration on the ultrasonic probe. Before performing time calibration, the electronic device can delete the identified second noise image from the ultrasonic image sequence. It should be noted that when deleting the second noise image before time calibration, the pose matrices corresponding to the second noise image in the ultrasonic probe pose sequence and the calibration model pose sequence will also be discarded.

[0053] In the embodiment of the present application, time calibration is to solve the problem of inconsistent transmission rates between different ultrasonic devices and optical navigation devices. By performing time calibration, the rate difference between the two can be obtained, providing a basis for subsequent ultrasonic probe pose calibration.

[0054] In a specific implementation, after deleting the second noise image and its corresponding ultrasonic probe pose matrix and calibration model pose matrix from the ultrasonic image sequence, the electronic device can extract the ultrasonic probe position sequence from the ultrasonic probe pose sequence corresponding to the remaining ultrasonic images, and obtain multiple eigenvectors and corresponding multiple eigenvalues by calculating the covariance matrix of the position sequence. The electronic device can use the eigenvector corresponding to the maximum eigenvalue as the main direction of the ultrasonic probe position sequence. Then, calculate the projection of the ultrasonic probe position sequence in the main direction to obtain a projection sequence. By normalizing the projection points of the obtained projection sequence, the ultrasonic probe displacement signal can be obtained. In this way, combined with the ultrasonic image displacement signal calculated in the foregoing steps, the electronic device can calculate the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal, and this rate difference can be represented by T.

[0055] It should be noted that since the movement signal of the ultrasonic image is opposite to the actual movement direction of the ultrasonic probe, the range for normalizing the projection points of the above projection sequence should be opposite to the range for normalizing the foregoing intersection point sequence. For example, the range for normalizing the intersection point sequence is (1, 0), and the range for normalizing the projection points of the projection sequence during time calibration is (0, 1).

[0056] Based on the ultrasonic image displacement signal and the ultrasonic probe displacement signal, a line graph of the two displacement signals can be obtained, and the electronic device can calculate the rate difference between the two based on this line graph. Since the pose sequence output rate of the optical navigation device is faster than the image sequence output rate of the ultrasonic device, when calculating the above rate difference, the ultrasonic image displacement signal can be used as a reference to move the ultrasonic probe displacement signal to align the ultrasonic probe displacement signal with the ultrasonic image displacement signal. Then, a target detection algorithm (single shot multibox detector, SSD) can be used to calculate the loss value between the aligned ultrasonic probe displacement signal and the ultrasonic image displacement signal, and the rate difference corresponding to the minimum loss value between the ultrasonic probe displacement signal and the ultrasonic image displacement signal is determined as the actual rate difference between the two. Thus, the time calibration of the ultrasonic probe is completed.

[0057] S105. After deleting the first noise image from the ultrasonic image sequence, perform spatial pose calibration on the ultrasonic probe based on the rate difference to obtain the spatial pose calibration matrix of the ultrasonic probe.

[0058] After completing the time calibration of the ultrasonic probe, the electronic device can perform spatial pose calibration on the ultrasonic probe. Before performing spatial pose calibration, the electronic device can delete the identified first noise image from the ultrasonic image sequence. Correspondingly, the pose sequences corresponding to the deleted first noise image in the ultrasonic probe pose sequence and the calibration model pose sequence will also be discarded.

[0059] In the embodiment of the present application, after deleting the first noise image and its corresponding ultrasonic probe pose matrix and calibration model pose matrix from the ultrasonic image sequence, the electronic device can delete the earliest obtained T pose sequences from the ultrasonic probe pose sequence and the calibration model pose sequence corresponding to the remaining ultrasonic images according to the rate difference T between the ultrasonic probe displacement signal and the ultrasonic image displacement signal, respectively obtaining the first transformation matrix and the second transformation matrix.

[0060] In addition, the electronic device can calculate the corresponding three-dimensional (3D) marker point sequence through processing such as similar triangles and coordinate transformation according to the calibration model and the 2D marker point sequence obtained by the foregoing 2D marker point recognition. The latest obtained T marker point groups in the 3D marker point sequence should also be deleted, so as to obtain the third transformation matrix. The 3D marker points in each of the above marker point groups belong to the same ultrasonic image.

[0061] On this basis, the electronic device can calculate the fourth transformation matrix, that is, the spatial pose calibration matrix, based on the first transformation matrix, the second transformation matrix, and the third transformation matrix, and complete the spatial pose calibration of the ultrasonic probe.

[0062] In a possible implementation manner of the embodiment of the present application, after calculating the fourth transformation matrix, that is, the spatial pose transformation matrix, the electronic device can further correct the spatial pose matrix.

[0063] In the embodiment of the present application, the electronic device can discard the noise data in the 3D marker point sequence, and then determine the target 3D marker point sequence based on the spatial pose calibration matrix and the 2D marker point sequence. The target 3D marker point sequence is also the theoretically obtained 3D marker point sequence through derivation. Then, calculate the error between the 3D marker point sequence after discarding the noise data and the above target 3D marker point sequence. The electronic device can correct the spatial pose calibration matrix according to the error. Specifically, if the above error is greater than the set error range, the electronic device can discard the marker point with the largest error in the 3D marker point sequence. At the same time, the 2D marker point corresponding to the discarded 3D marker point will also be discarded. Then, the fourth transformation matrix can be recalculated, and the above operation is repeated until the error is within the set error range. In this way, the finally corrected fourth transformation matrix is used as the spatial pose calibration matrix to complete the spatial pose calibration of the ultrasonic probe.

[0064] In another possible implementation manner of the embodiment of the present application, after completing the foregoing time calibration and spatial pose calibration, the calibration result can be further optimized.

[0065] Since the ultrasonic slice is 2D imaging, in the spatial pose calibration matrix, the accuracy of the attitude matrix is particularly important for the calibration result. A slight error in the rotation component will make the calibration result unstable, which is also a problem often encountered in actual use. Through calibration optimization in the embodiment of the present application, the attitude matrix can tend to be stable, and the stability of the calibration result can be improved.

[0066] In the embodiment of the present application, the fourth transformation matrix can be optimized according to the above 2D marker point sequence and 3D marker point sequence by using the modified Powell algorithm. The Powell algorithm is an algorithm for solving the local minimum of a function. Specifically, first, the search direction can be initialized, the initial search direction pool is set, the loss function f(x) is iteratively calculated, and the search direction is updated until the minimum value of f(x) meets the set threshold or the iteration period ends, and the search process is exited. Finally, the optimal solution of the fourth transformation matrix, that is, the optimized fourth transformation matrix, can be obtained. The electronic device can use the optimized fourth transformation matrix as the pose calibration matrix of the ultrasonic probe. At this time, the calibration of the ultrasonic probe is completed.

[0067] In the embodiments of the present application, the error in identifying marker points from ultrasonic images is reduced through image recognition; by screening and discarding the noise data in the collected data, the accuracy of subsequent processing results is ensured; by jointly calibrating the ultrasonic probe through time calibration and spatial pose calibration, the problem of inconsistent data transmission rates between different ultrasonic devices and optical navigation devices is solved, and it can ensure that the ultrasonic image sequence, ultrasonic probe pose sequence, and calibration model pose sequence required for calibration correspond one by one. Applying the calibration method provided by the embodiments of the present application can improve the calibration accuracy of the ultrasonic probe and the stability of the calibration results.

[0068] It should be noted that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The order of execution of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0069] For the convenience of understanding, the ultrasonic probe calibration method provided by the embodiments of the present application will be introduced below in combination with a complete example.

[0070] As Figure 2 shown, it is a schematic diagram of the calibration process of an ultrasonic probe provided by the embodiments of the present application. According to the calibration process Figure 2 shown, this method can be summarized into five parts, namely: data acquisition, image recognition, time calibration, spatial pose calibration, and calibration optimization. The following is a specific introduction.

[0071] 1. Data Acquisition In the data acquisition part, the electronic device can be connected to the optical navigation device and the ultrasonic device, and tracers can be installed on the ultrasonic probe and the calibration model. For example, a first tracer can be installed on the ultrasonic probe, and a second tracer can be installed on the calibration model. The above calibration model can adopt the multi-layer N-line model with acoustic wave reflection material at the bottom introduced above.

[0072] The optical navigation device should be placed within the optimal acquisition range. The multi-threaded acquisition function of the electronic device is started, and the ultrasonic probe is placed above the calibration model and slid. The electronic device can simultaneously acquire the ultrasonic probe pose sequence, calibration model pose sequence, and ultrasonic image sequence.

[0073] 2. Image Recognition In the image recognition part, the electronic device can perform marker point recognition and straight line recognition respectively according to the acquired two-dimensional ultrasonic image sequence.

[0074] 2.1. 2D Marker Point Recognition The 2D marker point recognition in the embodiments of this application refers to recognizing a 2D marker point sequence from a 2D ultrasound image sequence, and the 2D marker points are derived from the line segments in the calibration model. Due to the nature of ultrasound imaging, each marker point appears as an irregular small area in the image, so image processing and analysis need to be performed on each acquired 2D ultrasound image.

[0075] First, image preprocessing such as histogram equalization and threshold segmentation needs to be performed, and then connected component analysis of the image is carried out. The ROI region is recognized according to the characteristics of the connected components. Selecting the ROI region can improve the algorithm speed. By calculating the area and centroid of each connected component, the average area can be obtained, and then the connected components are screened according to the number of line segments in the calibration model to obtain multiple candidate connected components.

[0076] According to the centroid of the candidate connected components, with the horizontal direction as the search direction, collinearity analysis is performed on the centroids to obtain collinear connected components. There are a total of M groups of collinear connected components (N-line model with M layers), and each group of collinear connected components has three connected components. It is necessary to perform linear fitting on the centroids of the three connected components, and the obtained fitting points can be used as candidate 2D marker points.

[0077] Parallelism verification is performed on the candidate 2D marker point sequence, and the slope of each group of collinear connected components is calculated. If all the slope values are within a certain parallel range at this time, the parallelism verification is passed. If there is a slope value that does not meet the requirement, the corresponding ultrasound image is marked as the first noise image.

[0078] After passing the parallelism verification, distance verification needs to be performed, and the distance between each group of collinear connected components is calculated. If the distances of all the lines are within a certain range at this time, the distance verification is passed. If there is a distance that does not meet the requirement, the corresponding ultrasound image is marked as the first noise image.

[0079] After the time calibration is completed, when the spatial pose calibration is to be performed, according to the marked first noise images, the corresponding calibration model pose and ultrasound probe pose need to be discarded from the sequence.

[0080] 2.2. Line recognition Similar to the 2D marker point recognition, first, image preprocessing such as image denoising and contrast enhancement needs to be performed on the ultrasound image, and then the image is cropped in the horizontal direction. Maximum and minimum point analysis is performed on the obtained multiple cropped regions, and the coordinates of all the regional maximum and minimum points are recorded. Linear fitting is performed on the obtained set of regional maximum and minimum points. If the slope of the obtained line is within a certain range, the verification is passed; otherwise, the image is marked as the second noise image. Let the obtained line intersect with the vertical midline of the image to obtain an intersection point sequence. According to the image sequence, normalization processing is performed on each intersection point in the intersection point sequence with the vertical direction of the image as the reference, and the range is (1, 0), and then the displacement signal of the ultrasound image can be obtained.

[0081] During time calibration, according to the second noise image to be marked, the corresponding ultrasonic probe poses need to be discarded from the sequence.

[0082] 3. Time Calibration Time calibration is to solve the problem of inconsistent transmission rates between different ultrasonic devices and optical navigation devices. By performing time calibration, the rate difference between the two can be obtained, providing a basis for subsequent ultrasonic probe pose calibration. The rate difference between the ultrasonic device and the optical navigation device can be reflected by the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal.

[0083] Specifically, according to the ultrasonic probe pose sequence obtained by the optical navigation device, the position sequence of the ultrasonic probe can be extracted, the covariance matrix of the position sequence can be calculated, and the eigenvector and eigenvalue can be obtained. The eigenvector corresponding to the maximum eigenvalue is used as the main direction of the position sequence, and then the projection sequence of the position sequence on the main direction can be calculated. By normalizing the projection points of the projection sequence, the displacement signal of the ultrasonic probe can be obtained. Since the moving signal of the image is opposite to the actual probe moving direction, the range of normalization processing for the projection points is opposite to that for the intersection sequence, and the range of normalization processing for the projection points is (0, 1).

[0084] Based on the displacement signal of the ultrasonic image and the displacement signal of the ultrasonic probe, a line graph of the two displacement signals can be obtained. Since the pose output rate of the optical navigation device is faster than the image output rate of the ultrasonic device, taking the image displacement signal as the reference, the ultrasonic probe displacement signal is moved to align with the image displacement signal, and the SSD is used as the loss evaluation value. When the loss value is the smallest, the rate difference between the two displacement signals can be obtained, that is, the time calibration is completed.

[0085] 4. Spatial Pose Calibration In the part of spatial pose calibration, according to the identified 2D marker point sequence and the calibration model, the 3D marker point sequence can be calculated through similar triangles and coordinate transformation.

[0086] According to the rate difference T obtained from time calibration, the T earliest acquired data are deleted from the ultrasonic probe pose sequence to obtain the first transformation matrix; the same T earliest acquired data are deleted from the calibration model pose sequence to obtain the second transformation matrix; the T latest acquired data are deleted from the 2D marker point sequence and the 3D marker point sequence to obtain the third transformation matrix. Based on the first transformation matrix, the second transformation matrix, and the third transformation matrix, the spatial pose calibration matrix of the ultrasonic probe can be obtained, which is the fourth transformation matrix. On this basis, the fourth transformation matrix can also be corrected. Specifically, the noise data in the 3D marker point sequence can be discarded, and the derived 3D marker point sequence, i.e., the aforementioned target 3D marker point sequence, can be calculated using the fourth transformation matrix and the 2D marker point sequence. The 3D marker point sequence after discarding the noise data can be used to calculate the error with the above-mentioned target 3D marker point sequence. For example, the CR error can be used to calculate the difference between the two sequences. If the CR error value is greater than a certain range, the marker point with the largest error in the 3D marker point sequence can be discarded, and the corresponding 2D marker point should also be discarded. At this time, the fourth transformation matrix needs to be recalculated until the CR error value meets the set threshold.

[0087] 5. Calibration Optimization Since ultrasonic slicing is 2D imaging, in the spatial pose calibration matrix, the accuracy of the attitude matrix is particularly important for the calibration result. A slight error in the rotation component will make the calibration result unstable, which is also a problem often encountered in actual use. Through calibration optimization, the attitude matrix can be made to tend to be stable.

[0088] According to the above-mentioned 2D marker point sequence and 3D marker point sequence, the modified Powell method can be used to optimize the fourth transformation matrix. First, initialize the search direction, set the initial search direction pool, iteratively calculate the loss function f(x), and update the search direction until the minimum value of f(x) meets the set threshold or the iteration cycle ends, and then exit the search process. Finally, the optimal solution of the fourth transformation matrix can be obtained.

[0089] So far, the calibration of the ultrasonic probe has been completed.

[0090] Combined with the various embodiments described above, the embodiments of the present application also provide a calibration system for an ultrasonic probe. As Figure 3 shown, it is a schematic diagram of the calibration system for the ultrasonic probe provided by the embodiments of the present application. Figure 3 The calibration system 300 in it may include a calibration model 310, an electronic device 320, and an optical navigation device 330 and an ultrasonic device 340 connected to the electronic device 320. The calibration model 310 is a multi-layer N-line model with a sound wave reflection material at the bottom. The ultrasonic device 340 includes an ultrasonic probe 341. A first tracer and a second tracer are respectively installed on the ultrasonic probe 341 and the calibration model 310; where: The optical navigation device 330 can be used to obtain the ultrasonic probe pose sequence and the calibration model pose sequence by identifying the first tracer and the second tracer; The ultrasonic device 340 can be used to collect ultrasonic images during the process of placing the ultrasonic probe 341 on the calibration model 310 and sliding it; The electronic device 320 can be used to implement each step in the foregoing method embodiments. Exemplarily, the electronic device 320 can receive the ultrasonic probe pose sequence and the calibration model pose sequence obtained by the optical navigation device by identifying the first tracer and the second tracer; receive the ultrasonic image sequence obtained by the ultrasonic device 340 by collecting ultrasonic images; determine the first noise image and the second noise image in the ultrasonic image sequence by performing marker point recognition and straight line recognition on the ultrasonic image sequence; after deleting the second noise image from the ultrasonic image sequence, perform time calibration on the ultrasonic probe 341 to obtain the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal; after deleting the first noise image from the ultrasonic image sequence, perform spatial pose calibration on the ultrasonic probe 341 based on the rate difference to obtain the spatial pose calibration matrix of the ultrasonic probe 341, and complete the calibration of the ultrasonic probe 341.

[0091] For Figure 3 For the respective components of the calibration system 300 shown and their achievable functions, reference may be made to the introduction in the foregoing method embodiment section, and details are not described herein again.

[0092] Referring to Figure 4 , a schematic diagram of a calibration device for an ultrasonic probe provided by an embodiment of the present application is shown, which may specifically include a pose sequence receiving module 401, an image sequence receiving module 402, a noise image recognition module 403, a time calibration module 404, and a spatial calibration module 405, where: The pose sequence receiving module 401 is configured to receive the ultrasonic probe pose sequence and the calibration model pose sequence obtained by the optical navigation device by identifying the first tracer and the second tracer. The first tracer is mounted on the ultrasonic probe, and the second tracer is mounted on the calibration model. The calibration model is a multi-layer N-line model with a sound wave reflection material at the bottom; The image sequence receiving module 402 is configured to receive the ultrasonic image sequence obtained by the ultrasonic device by collecting ultrasonic images. The ultrasonic images are collected during the process of placing the ultrasonic probe on the calibration model and sliding; The noise image recognition module 403 is configured to determine the first noise image and the second noise image in the ultrasonic image sequence by performing marker point recognition and straight line recognition on the ultrasonic image sequence. The first noise image is an image in the ultrasonic image sequence that does not meet the marker point recognition requirements, and the second noise image is an image in the ultrasonic image sequence that does not meet the straight line recognition requirements; The time calibration module 404 is configured to perform time calibration on the ultrasonic probe after deleting the second noise image from the ultrasonic image sequence to obtain the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal; A spatial calibration module 405, configured to perform spatial pose calibration on the ultrasonic probe based on the rate difference after deleting the first noise image from the ultrasonic image sequence, so as to obtain a spatial pose calibration matrix of the ultrasonic probe.

[0093] In an embodiment of the present application, the marker points in the ultrasonic image sequence are used to represent line segments in the calibration model. Specifically, the noise image recognition module 403 may be configured to: Perform connected component analysis on each ultrasonic image in the ultrasonic image sequence based on the number of line segments in the calibration model, obtain multiple collinear connected component groups, and determine multiple candidate marker points based on the multiple collinear connected component groups; Perform a first verification on the multiple candidate marker points, and identify the ultrasonic image corresponding to the candidate marker points that do not meet the first verification requirement as the first noise image; Crop each ultrasonic image in the ultrasonic image sequence in the horizontal direction to obtain multiple cropped regions of each ultrasonic image, and determine the region maximum point corresponding to the maximum pixel value in each cropped region of each ultrasonic image; Perform a second verification on the corresponding ultrasonic image based on the region maximum points in each cropped region, and identify the ultrasonic image that does not meet the second verification requirement as the second noise image.

[0094] In a possible implementation manner of an embodiment of the present application, the noise image recognition module 403 may further be configured to: Identify multiple connected components included in each ultrasonic image in the ultrasonic image sequence, and screen out multiple candidate connected components with the same number according to the number of line segments in the calibration model; Perform collinearity analysis on the multiple candidate connected components to obtain multiple collinear connected component groups, and the number of the multiple collinear connected component groups is the same as the number of layers of the calibration model; Perform linear fitting on the centroids of the connected components in each collinear connected component group respectively, and determine the obtained fitting points as candidate marker points.

[0095] In another possible implementation manner of an embodiment of the present application, the first verification includes parallelism verification and distance verification. The noise image recognition module 403 may further be configured to: Calculate the first slope of the first straight line fitted by the multiple candidate marker points in each collinear connected component group respectively; if the first slope is within a preset first slope range, it is determined that the ultrasonic image corresponding to the multiple candidate marker points in the collinear connected component group passes the parallelism verification; Calculate the distance between each of the collinear connected component groups; if the distance is within a preset distance range, determine that the ultrasonic images corresponding to the multiple candidate marker points in the collinear connected component group pass the distance verification; Identify the ultrasonic images that do not pass the parallelism verification or the distance verification as first noise images.

[0096] In another possible implementation manner of the embodiment of the present application, the noise image recognition module 403 may further be used for: Calculate the second slope of the second straight line obtained by fitting based on the regional extreme value points in each of the cropped regions; if the second slope is within a preset second slope range, determine that the corresponding ultrasonic image passes the second verification; Identify the ultrasonic images that do not pass the second verification as second noise images.

[0097] In yet another possible implementation manner of the embodiment of the present application, the noise image recognition module 403 may further be used for: For the ultrasonic images that pass the second verification, determine the intersection points of the second straight line obtained by fitting in each ultrasonic image and the vertical midline of the current ultrasonic image, and obtain an intersection point sequence corresponding to the multiple ultrasonic images; Taking the image vertical direction as a reference, perform normalization processing on each intersection point in the intersection point sequence to obtain an ultrasonic image displacement signal.

[0098] In the embodiment of the present application, the time calibration module 404 may specifically be used for: After deleting the second noise images from the ultrasonic image sequence, extract an ultrasonic probe position sequence from the ultrasonic probe pose sequences corresponding to the remaining ultrasonic images; By calculating the covariance matrix of the position sequence, obtain a plurality of eigenvectors and corresponding eigenvalues, and use the eigenvector corresponding to the largest eigenvalue as the main direction of the ultrasonic probe position sequence; Calculate the projection of the ultrasonic probe position sequence in the main direction to obtain a projection sequence, and perform normalization processing on the projection sequence to obtain an ultrasonic probe displacement signal; Calculate the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal.

[0099] In a possible implementation manner of the embodiment of the present application, the time calibration module 404 may further be used for: Taking the ultrasonic image displacement signal as a reference, move the ultrasonic probe displacement signal to align the ultrasonic probe displacement signal with the ultrasonic image displacement signal; Calculate the loss value between the aligned displacement signal of the ultrasonic probe and the displacement signal of the ultrasonic image by using the target detection algorithm SSD; Determine the rate difference between the displacement signal of the ultrasonic probe and the displacement signal of the ultrasonic image corresponding to the minimum loss value.

[0100] In the embodiment of the present application, the spatial pose calibration module 405 can specifically be used for: After deleting the first noise image from the ultrasonic image sequence, delete the earliest obtained T pose sequences from the ultrasonic probe pose sequence and the calibration model pose sequence corresponding to the remaining ultrasonic images, respectively, to obtain a first transformation matrix and a second transformation matrix; where T is equal to the rate difference; According to the calibration model and the two-dimensional marker point sequence obtained by marker point recognition, determine the three-dimensional marker point sequence, and delete the latest obtained T marker point groups from the three-dimensional marker point sequence to obtain a third transformation matrix, and the three-dimensional marker points in each marker point group belong to the corresponding same ultrasonic image; Based on the first transformation matrix, the second transformation matrix, and the third transformation matrix, calculate the spatial pose calibration matrix.

[0101] In a possible implementation manner of the embodiment of the present application, the spatial pose calibration module 405 can also be used for: Determine the target three-dimensional marker point sequence based on the spatial pose calibration matrix and the two-dimensional marker point sequence; After discarding the noise data from the three-dimensional marker point sequence, calculate the error between the three-dimensional marker point sequence after discarding the noise data and the target three-dimensional marker point sequence; Correct the spatial pose calibration matrix according to the error.

[0102] A calibration device for an ultrasonic probe provided by an embodiment of the present application can implement each step in the foregoing method embodiments when this device is applied.

[0103] For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For related parts, refer to the description in the method embodiment part.

[0104] Refer to Figure 5 , which shows a schematic diagram of an electronic device provided by an embodiment of the present application. As Figure 5 shown, the electronic device 500 in the embodiment of the present application includes: a processor 510, a memory 520, and a computer program 521 stored in the memory 520 and executable on the processor 510. When the processor 510 executes the computer program 521, it implements the steps in each of the foregoing embodiments of the ultrasonic probe calibration method, for exampleFigure 1 Steps S101 to S105 shown. Alternatively, when the processor 510 executes the computer program 521, the functions of each module / unit in the above device embodiments are implemented, for example Figure 4 the functions of the modules 401 to 405 shown.

[0105] Exemplarily, the computer program 521 can be divided into one or more modules / units. The one or more modules / units are stored in the memory 520 and executed by the processor 510 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments can be used to describe the execution process of the computer program 521 in the electronic device 500. For example, the computer program 521 can be divided into a pose sequence receiving module, an image sequence receiving module, a noise image recognition module, a time calibration module, and a space calibration module. The specific functions of each module are as follows: The pose sequence receiving module is configured to receive the ultrasonic probe pose sequence and the calibration model pose sequence obtained by the optical navigation device through identifying the first tracer and the second tracer. The first tracer is installed on the ultrasonic probe, and the second tracer is installed on the calibration model. The calibration model is a multi-layer N-line model with a sound wave reflection material at the bottom; The image sequence receiving module is configured to receive the ultrasonic image sequence obtained by the ultrasonic device through collecting ultrasonic images. The ultrasonic images are collected during the process of placing the ultrasonic probe on the calibration model and sliding; The noise image recognition module is configured to determine the first noise image and the second noise image in the ultrasonic image sequence by performing marker point recognition and straight line recognition on the ultrasonic image sequence. The first noise image is an image in the ultrasonic image sequence that does not meet the marker point recognition requirements, and the second noise image is an image in the ultrasonic image sequence that does not meet the straight line recognition requirements; The time calibration module is configured to perform time calibration on the ultrasonic probe after deleting the second noise image from the ultrasonic image sequence, to obtain the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal; The space calibration module is configured to perform space pose calibration on the ultrasonic probe based on the rate difference after deleting the first noise image from the ultrasonic image sequence, to obtain the space pose calibration matrix of the ultrasonic probe.

[0106] The electronic device 500 can be a device capable of implementing the functions of each step in the foregoing method embodiments. The electronic device 500 can be a device such as a desktop computer or a cloud server. The electronic device 500 may include, but is not limited to, a processor 510 and a memory 520. Those skilled in the art can understand that Figure 5 This is merely an example of the electronic device 500 and does not limit the electronic device 500. It may include more or fewer components than those shown in the figure, or combine certain components, or have different components. For example, the electronic device 500 may further include input / output devices, network access devices, a bus, etc.

[0107] The processor 510 can be a central processing unit (CPU), or can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0108] The memory 520 can be an internal storage unit of the electronic device 500, such as the hard disk or memory of the electronic device 500. The memory 520 can also be an external storage device of the electronic device 500, such as a plug-in hard disk equipped on the electronic device 500, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 520 can also include both the internal storage unit and the external storage device of the electronic device 500. The memory 520 is used to store the computer program 521 and other programs and data required by the electronic device 500. The memory 520 can also be used to temporarily store data that has been output or is to be output.

[0109] The embodiments of the present application also disclose an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the methods described in the foregoing embodiments are implemented.

[0110] The embodiments of the present application also disclose a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a computer, the methods described in the foregoing various embodiments are implemented.

[0111] The embodiments of the present application also disclose a computer program product, including a computer program, and when the computer program runs on a computer, the computer is caused to execute the methods described in the foregoing various embodiments.

[0112] The foregoing embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A calibration method for an ultrasonic probe, characterized in that, Including: Receiving an ultrasonic probe pose sequence and a calibration model pose sequence obtained by an optical navigation device through identifying a first tracer and a second tracer, where the first tracer is mounted on the ultrasonic probe, the second tracer is mounted on the calibration model, and the calibration model is a multi-layer N-line model with acoustic wave reflection material at the bottom; Receiving an ultrasonic image sequence obtained by an ultrasonic device through collecting ultrasonic images, where the ultrasonic images are collected during the process of placing the ultrasonic probe on the calibration model and sliding; Determining a first noise image and a second noise image in the ultrasonic image sequence by performing marker point recognition and straight line recognition on the ultrasonic image sequence, where the first noise image is an image in the ultrasonic image sequence that does not meet the marker point recognition requirements, and the second noise image is an image in the ultrasonic image sequence that does not meet the straight line recognition requirements; After deleting the second noise image from the ultrasonic image sequence, performing time calibration on the ultrasonic probe to obtain a rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal; After deleting the first noise image from the ultrasonic image sequence, performing spatial pose calibration on the ultrasonic probe based on the rate difference to obtain a spatial pose calibration matrix of the ultrasonic probe.

2. The method according to claim 1, characterized in that, The marker points in the ultrasonic image sequence are used to represent the line segments in the calibration model. The determining the first noise image and the second noise image in the ultrasonic image sequence by performing marker point recognition and straight line recognition on the ultrasonic image sequence includes: Based on the number of line segments in the calibration model, performing connected component analysis on each ultrasonic image in the ultrasonic image sequence to obtain multiple collinear connected component groups, and determining multiple candidate marker points based on the multiple collinear connected component groups; By performing a first verification on the multiple candidate marker points, identifying the ultrasonic image corresponding to the candidate marker point that does not meet the first verification requirements as the first noise image; Cropping each ultrasonic image in the ultrasonic image sequence in the horizontal direction to obtain multiple cropped regions of each ultrasonic image, and determining a region maximum point corresponding to the maximum pixel value in each cropped region of each ultrasonic image; Performing a second verification on the corresponding ultrasonic image based on the region maximum points in each cropped region, and identifying the ultrasonic image that does not meet the second verification requirements as the second noise image.

3. The method according to claim 2, wherein The based on the number of line segments in the calibration model, performing connected component analysis on each ultrasonic image in the ultrasonic image sequence to obtain multiple collinear connected component groups, and determining multiple candidate marker points based on the multiple collinear connected component groups includes: Identifying multiple connected components included in each ultrasonic image in the ultrasonic image sequence, and screening to obtain the same number of multiple candidate connected components according to the number of line segments in the calibration model; Performing collinearity analysis on the multiple candidate connected components to obtain multiple collinear connected component groups, and the number of the multiple collinear connected component groups is the same as the number of layers of the calibration model; Performing straight line fitting on the centroids of the connected components in each collinear connected component group respectively, and determining the obtained fitting points as candidate marker points.

4. The method according to claim 2, wherein The first verification includes parallelism verification and distance verification. By performing the first verification on multiple candidate marker points, identifying the ultrasonic images corresponding to the candidate marker points that do not meet the requirements of the first verification as first noise images includes: Calculating the first slope of the first straight line fitted by multiple candidate marker points in each collinear connected region group respectively; if the first slope is within a preset first slope range, determining that the ultrasonic images corresponding to the multiple candidate marker points in the collinear connected region group pass the parallelism verification; Calculating the distances between each collinear connected region group respectively; if the distances are within a preset distance range, determining that the ultrasonic images corresponding to the multiple candidate marker points in the collinear connected region group pass the distance verification; Identifying the ultrasonic images that do not pass the parallelism verification or the distance verification as first noise images.

5. The method according to any one of claims 2 to 4, characterized in that Based on the regional extreme value points in each cropping region, performing a second verification on the corresponding ultrasonic images, and identifying the ultrasonic images that do not meet the requirements of the second verification as second noise images, including: Calculating the second slope of the second straight line fitted based on the regional extreme value points in each cropping region; if the second slope is within a preset second slope range, determining that the corresponding ultrasonic image passes the second verification; Identifying the ultrasonic images that do not pass the second verification as second noise images.

6. The method according to claim 5, characterized in that, It further includes: For the ultrasonic images that pass the second verification, determining the intersection points of the second straight line fitted in each ultrasonic image and the vertical midline of the current ultrasonic image, and obtaining an intersection point sequence corresponding to multiple ultrasonic images; Taking the image vertical direction as a reference, performing normalization processing on each intersection point in the intersection point sequence to obtain an ultrasonic image displacement signal.

7. The method according to any one of claims 1 to 4 or 6, characterized in that, After deleting the second noise images from the ultrasonic image sequence, performing time calibration on the ultrasonic probe to obtain the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal, including: After deleting the second noise images from the ultrasonic image sequence, extracting the ultrasonic probe position sequence from the ultrasonic probe pose sequence corresponding to the remaining ultrasonic images; By calculating the covariance matrix of the position sequence, obtaining multiple eigenvectors and corresponding multiple eigenvalues, and taking the eigenvector corresponding to the maximum eigenvalue as the main direction of the ultrasonic probe position sequence; Calculating the projection of the ultrasonic probe position sequence in the main direction to obtain a projection sequence, and performing normalization processing on the projection points of the projection sequence to obtain an ultrasonic probe displacement signal; Calculating the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal.

8. The method according to claim 7, characterized in that Calculating the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal includes: Taking the ultrasonic image displacement signal as a reference, moving the ultrasonic probe displacement signal to align the ultrasonic probe displacement signal with the ultrasonic image displacement signal; Using the target detection algorithm SSD to calculate the loss value of the aligned ultrasonic probe displacement signal and the ultrasonic image displacement signal; Determine the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal corresponding to the minimum of the loss value.

9. The method according to any one of claims 1 to 4, 6 or 8, characterized in that After deleting the first noise image from the ultrasonic image sequence, based on the rate difference, perform spatial pose calibration on the ultrasonic probe to obtain the spatial pose calibration matrix of the ultrasonic probe, including: After deleting the first noise image from the ultrasonic image sequence, delete the earliest obtained T pose sequences from the ultrasonic probe pose sequence and the calibration model pose sequence corresponding to the remaining ultrasonic images, respectively, to obtain a first transformation matrix and a second transformation matrix; where the value of T is equal to the rate difference; According to the calibration model and the two-dimensional marker point sequence obtained by marker point recognition, determine the three-dimensional marker point sequence, and delete the latest obtained T marker point groups from the three-dimensional marker point sequence, to obtain a third transformation matrix, and the three-dimensional marker points in each marker point group belong to the corresponding same ultrasonic image; Based on the first transformation matrix, the second transformation matrix, and the third transformation matrix, calculate to obtain the spatial pose calibration matrix.

10. The method according to claim 9, wherein It further includes: Based on the spatial pose calibration matrix and the two-dimensional marker point sequence, determine the target three-dimensional marker point sequence; After discarding the noise data from the three-dimensional marker point sequence, calculate the error between the three-dimensional marker point sequence after discarding the noise data and the target three-dimensional marker point sequence; According to the error, correct the spatial pose calibration matrix.

11. A calibration system for an ultrasonic probe, characterized in that, It includes a calibration model, an electronic device, an optical navigation device and an ultrasonic device connected to the electronic device. The calibration model is a multi-layer N-line model with a sound wave reflection material at the bottom. The ultrasonic device includes an ultrasonic probe, and a first tracer and a second tracer are respectively installed on the ultrasonic probe and the calibration model; where: The optical navigation device is used to obtain the ultrasonic probe pose sequence and the calibration model pose sequence by identifying the first tracer and the second tracer; The ultrasonic device is used to collect ultrasonic images during the process of placing the ultrasonic probe on the calibration model and sliding; The electronic device is used to implement the method described in any one of claims 1 to 10.

12. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it enables the electronic device to implement the method described in any one of claims 1 to 10.

Images