Ultrasonic probe calibration method, calibration system and electronic equipment

Abstract

The embodiments of the present application are applicable to the fields of medical imaging technology and image calibration technology, and provide a calibration method, calibration system, and electronic equipment for an ultrasound probe. The method comprises: receiving an ultrasound probe pose sequence and a calibration model pose sequence; receiving an ultrasound image sequence; determining a first noise image and a second noise image in the ultrasound image sequence by performing marker point recognition and line recognition on the ultrasound image sequence; after deleting the second noise image from the ultrasound image sequence, performing time calibration on the ultrasound probe to obtain a rate difference between an ultrasound probe displacement signal and an ultrasound image displacement signal; after deleting the first noise image from the ultrasound image sequence, performing spatial pose calibration on the ultrasound probe based on the rate difference to obtain a spatial pose calibration matrix of the ultrasound probe. The above method can improve the calibration accuracy of the ultrasound probe and the stability of the calibration results.

Description

Technical Field

[0001] The embodiments of the present application belong to the fields of medical imaging technology and image calibration technology, and in particular, relate to a calibration method, calibration system, and electronic equipment for an ultrasound probe. Background Art

[0002] With the development of medical imaging, image-guided technology is becoming increasingly important in surgical operations. Real-time intraoperative imaging and precise positioning are the key to successful surgery. Ultrasound imaging, as a safe and non-invasive imaging technology, has been widely used in many surgical fields, especially in orthopedics, neurosurgery, tumor surgery, cardiac surgery and other fields. Imaging technology based on robot-assisted navigation has significant advantages in surgical operations, especially in minimally invasive surgery. Compared with traditional surgery, the application of this technology can effectively improve the success rate and safety of surgery. Commonly used robot-assisted navigation imaging technologies include X-ray imaging, CT imaging, ultrasound imaging, etc.

[0003] Radiographic imaging technologies such as X-rays and CT scans generate static image data. Using rigid and nonrigid transformations, static image data can be used to register preoperative and intraoperative patient images, or to register the patient's actual position with intraoperative images. This registration method offers high accuracy and stability and is widely used in surgical navigation. However, the use of radiographic imaging technologies can expose patients and physicians to significant radiation exposure.

[0004] Ultrasound imaging technology does not cause radiation to patients or doctors. During intraoperative ultrasound imaging, the doctor usually holds an ultrasound probe and scans the area or part of the patient requiring surgery, obtaining a dynamic two-dimensional (2D) ultrasound image sequence. The resulting 2D image sequence is then aligned with the preoperative image or the patient's actual position during surgery to achieve the purpose of intraoperative ultrasound guidance. The accuracy of this alignment method depends on the calibration of the ultrasound probe. Because the calibration process of the ultrasound probe is cumbersome and complex, the stability of the calibration results is poor, resulting in the seldom application of ultrasound imaging technology in surgical navigation. Summary of the Invention

[0005] In view of this, an embodiment of the present application provides an ultrasound probe calibration method, a calibration system, and an electronic device, which are used together to calibrate the ultrasound probe by combining time calibration and space calibration, thereby improving the calibration accuracy of the ultrasound probe and the stability of the calibration results.

[0006] A first aspect of an embodiment of the present application provides a method for calibrating an ultrasound probe, comprising:

[0007] receiving an ultrasound probe pose sequence and a calibration model pose sequence obtained by an optical navigation device by identifying a first tracer and a second tracer, wherein the first tracer is mounted on the ultrasound probe, and the second tracer is mounted on the calibration model, wherein the calibration model is a multi-layer N-line model with a sound wave reflecting material on the bottom;

[0008] receiving an ultrasound image sequence obtained by an ultrasound device through acquisition of ultrasound images, wherein the ultrasound images are acquired by placing the ultrasound probe on the calibration model and sliding it;

[0009] Determining a first noise image and a second noise image in the ultrasound image sequence by performing marker point recognition and line recognition on the ultrasound image sequence, wherein the first noise image is an image in the ultrasound image sequence that does not meet the marker point recognition requirement, and the second noise image is an image in the ultrasound image sequence that does not meet the line recognition requirement;

[0010] After deleting the second noise image from the ultrasonic image sequence, performing time calibration on the ultrasonic probe to obtain a rate difference between an ultrasonic probe displacement signal and an ultrasonic image displacement signal;

[0011] After deleting the first noise image from the ultrasound image sequence, the ultrasound probe is spatially calibrated based on the rate difference to obtain a spatial pose calibration matrix of the ultrasound probe.

[0012] A second aspect of the embodiments of the present application provides a calibration device for an ultrasound probe, comprising:

[0013] a pose sequence receiving module, configured to receive an ultrasound probe pose sequence and a calibration model pose sequence obtained by the optical navigation device by identifying a first tracer and a second tracer, wherein the first tracer is mounted on the ultrasound probe, and the second tracer is mounted on the calibration model, wherein the calibration model is a multi-layer N-line model having a sound wave reflecting material on the bottom;

[0014] An image sequence receiving module, configured to receive an ultrasound image sequence obtained by an ultrasound device through acquisition of ultrasound images, wherein the ultrasound images are acquired by placing the ultrasound probe on the calibration model and sliding the probe;

[0015] a noise image recognition module, configured to determine a first noise image and a second noise image in the ultrasound image sequence by performing marker point recognition and line recognition on the ultrasound image sequence, wherein the first noise image is an image in the ultrasound image sequence that does not meet the marker point recognition requirement, and the second noise image is an image in the ultrasound image sequence that does not meet the line recognition requirement;

[0016] a time calibration module, configured to perform time calibration on the ultrasound probe after deleting the second noise image from the ultrasound image sequence, to obtain a rate difference between the ultrasound probe displacement signal and the ultrasound image displacement signal;

[0017] A spatial calibration module is used to perform spatial posture calibration on the ultrasound probe based on the rate difference after deleting the first noise image from the ultrasound image sequence, so as to obtain a spatial posture calibration matrix of the ultrasound probe.

[0018] A third aspect of the embodiments of the present application provides an ultrasound probe calibration system, comprising a calibration phantom, an electronic device, and an optical navigation device and an ultrasound device connected to the electronic device. The calibration phantom is a multilayer N-line phantom having a sound wave reflecting material on the bottom. The ultrasound device includes an ultrasound probe. A first tracer and a second tracer are mounted on the ultrasound probe and the calibration phantom, respectively.

[0019] The optical navigation device is used to obtain an ultrasound probe pose sequence and a calibration model pose sequence by identifying the first tracer and the second tracer;

[0020] The ultrasound device is used to collect ultrasound images during the process of placing the ultrasound probe on the calibration model and sliding it;

[0021] The electronic device is used to implement the method described in the first aspect above.

[0022] A fourth aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the method described in the first aspect above.

[0023] A fifth aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a computer, the method described in the first aspect above is implemented.

[0024] A sixth aspect of the embodiments of the present application provides a computer program product, including a computer program, which, when executed, enables the method described in the first aspect to be executed.

[0025] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0026] The embodiments of the present application reduce the errors in identifying marker points from ultrasound images through image recognition; by screening and discarding noise data in the collected data, the accuracy of subsequent processing results is guaranteed; by combining time calibration and spatial posture calibration to calibrate the ultrasound probe, the problem of inconsistent data transmission rates between different ultrasound devices and optical navigation devices is solved, and it can ensure that the ultrasound image sequence required for calibration, the ultrasound probe posture sequence, and the calibration model posture sequence correspond one to one. Applying the calibration method provided in the embodiments of the present application can improve the calibration accuracy of the ultrasound probe and the stability of the calibration results. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0028] Figure 1 is a schematic diagram of a calibration method for an ultrasound probe provided in an embodiment of the present application;

[0029] Figure 2 is a schematic diagram of a calibration process of an ultrasound probe provided in an embodiment of the present application;

[0030] Figure 3 is a schematic diagram of a calibration system for an ultrasound probe provided in an embodiment of the present application;

[0031] Figure 4 is a schematic diagram of a calibration device for an ultrasound probe provided in an embodiment of the present application;

[0032] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present application with unnecessary details.

[0034] As introduced in the background technology, intraoperative ultrasound imaging is mainly performed by the doctor holding a handheld ultrasound probe to scan the area or part of the patient who needs surgery, and then using the dynamic 2D ultrasound image sequence obtained by the scan to align with the preoperative image or the actual position of the patient during the operation, so as to achieve the purpose of intraoperative ultrasound guidance. The calibration of the ultrasound probe in the related art is only the spatial posture calibration. Imaging errors such as ghosting, missing images, and afterimages will inevitably occur during the scanning process. In addition, the inconsistency of image processing and transmission rates between ultrasound devices, as well as the large difference in output rates of optical navigation devices, will also affect the calibration process. These problems may lead to large errors in the transformation matrix that needs to be calculated during the calibration process, which in turn leads to larger calibration errors.

[0035] To address the above problems, the embodiments of the present application provide an ultrasound probe calibration method, device, and electronic device, which can be combined with time calibration and space calibration to calibrate the ultrasound probe, thereby improving the calibration accuracy of the ultrasound probe and the stability of the calibration results.

[0036] The technical solution of this application is described below through specific embodiments.

[0037] Reference Figure 1 , which shows a schematic diagram of a calibration method for an ultrasound probe provided in an embodiment of the present application, which may specifically include the following steps:

[0038] S101 : Receive an ultrasound probe pose sequence and a calibration model pose sequence obtained by an optical navigation device through identifying a first tracer and a second tracer.

[0039] It should be noted that the embodiments of the present application can be applied to electronic devices, that is, the execution subject of the present method can be an electronic device. By executing the various steps of the method provided in the embodiments of the present application, the electronic device can combine time calibration and spatial calibration to calibrate the ultrasound probe, thereby obtaining a calibration result with high accuracy and high stability. The above-mentioned electronic device can be a desktop computer, a cloud server, or other device. The embodiments of the present application do not limit the specific type of electronic device.

[0040] During ultrasound probe calibration, a calibration phantom can be used. The calibration phantom used in the embodiments of the present application can be a multi-layer N-line phantom with a sound-reflecting material at the bottom. Within each layer of the calibration phantom, a set of parallel line segments can be formed by stringing. By connecting the two endpoints of the parallel line segments again with strings, a pattern resembling the letter "N" can be formed, representing an N-line. Conventional N-line phantoms are typically three-layered without a bottom surface. In the embodiments of the present application, the N-line phantom uses sound-reflecting material to create a bottom surface. During ultrasound probe calibration, the sound-reflecting material on the bottom surface reflects sound waves, helping to achieve clearer ultrasound images. The number of layers in the N-line phantom in the embodiments of the present application can be determined based on the actual requirements of the area to be scanned by the ultrasound device. Generally, the number of layers in the N-line phantom is positively correlated with the depth of the area to be scanned. For example, for an area to be scanned at a depth of approximately 10 cm, a 3-5-layer N-line phantom can be used to calibrate the ultrasound probe. For areas to be scanned at depths of 15 cm, 20 cm, or even deeper, an 8- or more-layer N-line phantom can be used to calibrate the ultrasound probe.

[0041] In an embodiment of the present application, markers can be installed on both the ultrasound probe and the calibration model. For example, the two markers can be distinguished by a first marker and a second marker. The first marker can be installed on the ultrasound probe, and the second marker can be installed on the calibration model. In this way, during the actual calibration process, the optical navigation device can obtain the ultrasound probe pose sequence and the calibration model pose sequence by identifying the two markers.

[0042] Both the ultrasound probe pose sequence and the calibration model pose sequence include multiple pose matrices. Each pose matrix can be decomposed into a pose matrix and a position matrix. Accordingly, the combination of multiple pose matrices can be called a pose sequence, and the combination of multiple position matrices can be called a position sequence.

[0043] In a specific implementation, before starting the electronic device and the optical navigation device, the optical navigation device can be placed within the optimal acquisition range through settings. This helps to more accurately identify each tracer and obtain a more accurate and complete pose sequence.

[0044] The ultrasound probe pose sequence and calibration model pose sequence obtained by the optical navigation device through identifying the first tracer and the second tracer can be transmitted to the electronic device, which can be further processed by the electronic device in combination with the ultrasound image sequence collected by the ultrasound device.

[0045] S102: Receive an ultrasound image sequence obtained by an ultrasound device through ultrasound image acquisition.

[0046] During the ultrasound probe calibration process, the ultrasound probe can be placed on the calibration model and slid. During this sliding process, the ultrasound device can capture ultrasound images. Each captured ultrasound image is a two-dimensional image. Successively captured ultrasound images form an ultrasound image sequence. The ultrasound image sequence can be transmitted from the ultrasound device to an electronic device.

[0047] In an embodiment of the present application, an electronic device is connected to an optical navigation device and an ultrasound device. When calibrating the ultrasound probe, the electronic device can enable a multi-threaded acquisition function, and simultaneously receive the ultrasound probe pose sequence and the calibration model pose sequence identified by the optical navigation device, as well as the ultrasound image sequence collected by the ultrasound device. The pose matrix in the above-mentioned ultrasound probe pose sequence, the pose matrix in the calibration model pose sequence, and the ultrasound image in the ultrasound image sequence have a corresponding relationship. For example, when the ultrasound probe slides on the calibration model, the ultrasound device will collect an ultrasound image. At this time, the optical navigation device can obtain the ultrasound probe pose matrix and the calibration model pose matrix by identifying the tracer on the ultrasound probe and the calibration model. The ultrasound image, the ultrasound probe pose matrix, and the calibration model pose matrix at this moment have a corresponding relationship.

[0048] S103 : Determine a first noise image and a second noise image in the ultrasound image sequence by performing marker point recognition and line recognition on the ultrasound image sequence.

[0049] After receiving data such as the ultrasound probe pose sequence, the calibration model pose sequence, and the ultrasound image pose sequence, the electronic device can process the received data to identify ultrasound images with imaging errors and ultrasound images that do not correspond to the pose sequence. In the embodiments of the present application, such images can be referred to as noise images.

[0050] In an embodiment of the present application, an ultrasound image can be processed by performing marker point recognition and line recognition on each ultrasound image contained therein, and determining images that do not meet the corresponding recognition requirements as noise images. Specifically, images in an ultrasound image sequence that do not meet the marker point recognition requirements can be marked as first noise images, and images in the sequence that do not meet the line recognition requirements can be marked as second noise images. The above-mentioned first noise image and second noise image will be discarded in the subsequent processing process. For example, when performing time calibration, the second noise image can be discarded; after completing time calibration, when performing spatial pose calibration, the first noise image can be discarded. When the first noise image and the second noise image are discarded, their corresponding ultrasound probe pose matrix and calibration model pose matrix will also be discarded from the ultrasound probe pose sequence or the calibration model pose sequence.

[0051] In an embodiment of the present application, the first noise image in the ultrasound image that does not meet the marker point recognition requirements can be determined by marker point recognition. The above-mentioned marker point recognition can refer to the recognition of two-dimensional (2D) marker points in the ultrasound image. The 2D marker points in the ultrasound image are derived from the line segments in the calibration model, that is, the line segments similar to the shape of the letter "N" introduced above, that is, the marker points in the ultrasound image sequence can be used to characterize the line segments in the calibration model. Since each layer of the calibration model contains a figure similar to the shape of the letter "N", accordingly, each layer of the calibration model includes 3 line segments. Therefore, the total number of line segments of the calibration model depends on the number of layers of the model. For example, assuming that the calibration model is a 6-layer N-line model, since each layer includes 3 line segments, the 6-layer N-line model has a total of 18 line segments.

[0052] Due to the nature of ultrasound imaging, each 2D marker in an ultrasound image appears as a small, irregular region. Therefore, each acquired ultrasound image can be processed and analyzed to identify the 2D markers it contains.

[0053] In a possible implementation of an embodiment of the present application, when performing marker point recognition on an ultrasound image sequence and determining the first noise image in the sequence, a connected domain analysis can be performed on each ultrasound image in the ultrasound image sequence based on the number of line segments in the calibration model to obtain multiple collinear connected domain groups, and based on the multiple collinear connected domain groups, multiple candidate marker points can be determined.

[0054] In an embodiment of the present application, the electronic device may first pre-process each ultrasound image, such as performing histogram equalization and threshold segmentation on the ultrasound image. Connected domain analysis may then be performed on the image to identify multiple connected domains contained in each ultrasound image in the ultrasound image sequence, and the same number of candidate connected domains may be obtained by screening based on the number of line segments in the calibration model.

[0055] Specifically, the electronic device performs a connected domain analysis on the ultrasound image to obtain multiple connected domains contained in each image. It can then identify the region of interest (ROI) based on the characteristics of the connected domains. The algorithm speed can be improved by selecting the ROI area. Then, by calculating the area and centroid of each connected domain, the area average can be obtained. The identified connected domains can be screened according to the number of line segments of the calibration model to obtain candidate connected domains. For example, assuming that the calibration model is a 6-layer N-line model with a total of 18 line segments. Therefore, the connected domains identified in the ultrasound image should theoretically include 18. Assuming that 24 connected domains are actually identified, these 24 connected domains can be sorted according to the area size of each connected domain. Based on the area average, 9 connected domains on the left and 9 on the right are selected, and these 18 selected connected domains are used as candidate connected domains. That is, the electronic device can calculate the area of ​​each identified connected domain, and sort each connected domain according to the size of the area. Based on the average area value, multiple connected domains are selected in order of area size as candidate connected domains, so that the number of selected candidate connected domains is equal to the number of line segments of the calibration model.

[0056] After screening multiple candidate connected domains, the electronic device can perform collinearity analysis on the multiple candidate connected domains to obtain multiple collinear connected domain groups. The number of these multiple collinear connected domain groups is the same as the number of layers in the calibration model. For example, when using a 6-layer N-line model for calibration, the number of screened collinear connected domain groups should be 6. The electronic device can then perform linear fitting on the centroids of the connected domains in each collinear connected domain group and determine the resulting fitting points as candidate marker points.

[0057] Specifically, the electronic device can perform collinearity analysis on each candidate connected domain's centroid, with the horizontal direction as the search direction, to obtain collinear connected domains, which are then formed into collinear connected domain groups. Each collinear connected domain group includes three collinear connected domains. Linear fitting is performed on the centroids of these three collinear connected domains, and the resulting fitted points can be used as candidate 2D marker points.

[0058] After completing the 2D marker point recognition, the electronic device may verify the multiple candidate marker points. By verifying the multiple candidate marker points, the electronic device may identify the ultrasound images corresponding to the candidate marker points that do not meet the verification requirements as first noise images. This verification of the multiple candidate marker points may be referred to as a first verification. That is, by performing the first verification on the multiple candidate marker points, the electronic device may identify the ultrasound images corresponding to the candidate marker points that do not meet the first verification requirements as first noise images.

[0059] In an embodiment of the present application, the first check may include a parallelism check and a distance check. The electronic device may first perform a parallelism check on the candidate marker points. Specifically, the electronic device may respectively calculate the slope of the straight line obtained by fitting multiple candidate marker points in each collinear connected domain group. For example, the straight line obtained by fitting multiple candidate marker points may be referred to as a first straight line, and accordingly, the slope of the straight line may be referred to as a first slope. If the above-mentioned first slope is within a preset first slope range, the electronic device may determine that the ultrasound images corresponding to the multiple candidate marker points in the collinear connected domain group pass the parallelism check.

[0060] For candidate markers that pass the parallelism check, the electronic device can perform a distance check on them. Specifically, the electronic device can calculate the distance between each collinear connected domain group. If the distance is within a preset distance range, it can be determined that the ultrasound images corresponding to the multiple candidate marker points in the collinear connected domain group pass the distance check.

[0061] The electronic device may identify an ultrasound image that fails the parallelism check or the distance check as a first noise image. When subsequent time calibration is completed and spatial pose calibration is required, the ultrasound image identified as the first noise image and its corresponding ultrasound probe pose matrix and calibration model pose matrix will be discarded from the image sequence and pose sequence.

[0062] After completing the recognition of the first noise image, the electronic device can determine a second noise image in the ultrasound image sequence through line recognition. The second noise image is also an ultrasound image that does not meet the above line recognition requirements.

[0063] When performing straight line recognition on an ultrasound image, each ultrasound image may also be preprocessed first, for example, image denoising, contrast enhancement, and other preprocessing may be performed on the ultrasound image. Then, each ultrasound image in the ultrasound image sequence may be cropped in the horizontal direction to obtain multiple cropped areas for each ultrasound image. In this way, each ultrasound image is cropped into multiple strip-shaped areas. On this basis, the electronic device may determine the regional maximum point corresponding to the pixel maximum value in each cropped area of ​​each ultrasound image, and record the coordinates of all regional maximum points. For an ultrasound image, the number of regional maximum points it includes is equal to the number of areas into which the ultrasound image is cropped. For example, if an ultrasound image is cropped into 8 areas, 8 regional maximum points may be obtained by determining the pixel maximum value in each cropped area.

[0064] The electronic device may then verify the corresponding ultrasound image based on the regional maximum point in each cropped region. To distinguish this from the aforementioned first verification, this verification may be referred to as a second verification. Ultrasound images that do not meet the second verification requirements may be identified as second noise images.

[0065] Specifically, the electronic device may calculate the slope of a straight line obtained by fitting the regional maximum points in each cropped area. For example, the straight line may be referred to as a second straight line, and accordingly, its slope may be a second slope. If the second slope is within a preset second slope range, it may be determined that the corresponding ultrasound image has passed the second verification. The electronic device may identify an ultrasound image that has failed the second verification as a second noise image.

[0066] In an embodiment of the present application, for ultrasound images that pass the second verification, that is, ultrasound images that are not identified as second noise images, the electronic device can determine the intersection of the second straight line fitted in each ultrasound image and the vertical center line of the current ultrasound image, and obtain a sequence of intersections corresponding to multiple ultrasound images.

[0067] Specifically, a straight line on the ultrasound image, i.e., the aforementioned second straight line, can be fitted based on the regional maximum points within each cropped region. Simultaneously, the electronic device can draw a vertical line, i.e., a vertical centerline, through the midpoint of the image on the ultrasound image. The vertical centerline and the fitted second straight line intersect to form an intersection point. The same process is performed on each ultrasound image that passes the second verification, resulting in an intersection point on each ultrasound image. These intersection points can be combined to form an intersection point sequence.

[0068] The electronic device can normalize each intersection point in the intersection point sequence based on the vertical direction of the image to obtain an ultrasonic image displacement signal. The displacement signal can represent the actual displacement of the ultrasonic image collected during the calibration process.

[0069] In the image coordinate system, the vertical direction of the image is the y-axis direction. Therefore, the above-mentioned normalization processing of each intersection point based on the vertical direction of the image can refer to normalizing the x-axis coordinate value in the coordinates of each intersection point, and the normalization processing range is (1,0).

[0070] S104 : After deleting the second noise image from the ultrasonic image sequence, perform time calibration on the ultrasonic probe to obtain a rate difference between an ultrasonic probe displacement signal and an ultrasonic image displacement signal.

[0071] After completing steps S101-S103, the electronic device can perform time calibration on the ultrasound probe. Before performing time calibration, the electronic device can delete the identified second noise image from the ultrasound image sequence. It should be noted that when the second noise image is deleted before time calibration, the pose matrix corresponding to the second noise image in the ultrasound probe pose sequence and the calibration model pose sequence will also be discarded.

[0072] In the embodiment of the present application, time calibration is performed to solve the problem of inconsistent transmission rates between different ultrasound devices and optical navigation devices. The rate difference between the two can be obtained through time calibration, providing a basis for subsequent ultrasound probe posture calibration.

[0073] In a specific implementation, after deleting the second noise image and its corresponding ultrasound probe pose matrix and calibration model pose matrix from the ultrasound image sequence, the electronic device can extract the ultrasound probe position sequence from the ultrasound probe pose sequence corresponding to the remaining ultrasound images, and obtain multiple eigenvectors and corresponding multiple eigenvalues ​​by calculating the covariance matrix of the position sequence. The electronic device can use the eigenvector corresponding to the maximum eigenvalue as the main direction of the ultrasound probe position sequence. Then, the projection of the ultrasound probe position sequence in the main direction is calculated to obtain a projection sequence, and the ultrasound probe displacement signal can be obtained by normalizing the projection points of the obtained projection sequence. In this way, combined with the ultrasound image displacement signal calculated in the above steps, the electronic device can calculate the rate difference between the ultrasound probe displacement signal and the ultrasound image displacement signal, and the rate difference can be represented by T.

[0074] It should be noted that because the motion signal of the ultrasound image is opposite to the actual motion direction of the ultrasound probe, the normalization range for the projection points of the above projection sequence should be opposite to the normalization range for the intersection sequence mentioned above. For example, the normalization range for the intersection sequence mentioned above is (1, 0), and the normalization range for the projection points of the projection sequence during time calibration is (0, 1).

[0075] According to the ultrasonic image displacement signal and the ultrasonic probe displacement signal, a broken line graph of the two displacement signals can be obtained, and the electronic device can calculate the rate difference between the two based on the broken line graph. Since the posture sequence output rate of the optical navigation device is faster than the image sequence output rate of the ultrasonic device, when calculating the above rate difference, the ultrasonic image displacement signal can be used as a reference to move the ultrasonic probe displacement signal to align the ultrasonic probe displacement signal with the ultrasonic image displacement signal. Then, the target detection algorithm (single shot multibox detector, SSD) can be used to calculate the loss value of the aligned ultrasonic probe displacement signal and the ultrasonic image displacement signal, and the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal corresponding to the minimum loss value is determined as the actual rate difference between the two. At this point, the time calibration of the ultrasonic probe is completed.

[0076] S105 . After deleting the first noise image from the ultrasound image sequence, perform spatial pose calibration on the ultrasound probe based on the rate difference to obtain a spatial pose calibration matrix of the ultrasound probe.

[0077] After completing the temporal calibration of the ultrasound probe, the electronic device can perform spatial pose calibration on the ultrasound probe. Before performing spatial pose calibration, the electronic device can delete the identified first noise image from the ultrasound image sequence. Accordingly, the pose sequences corresponding to the deleted first noise image in the ultrasound probe pose sequence and the calibration model pose sequence will also be discarded.

[0078] In an embodiment of the present application, after deleting the first noise image and its corresponding ultrasound probe pose matrix and calibration model pose matrix from the ultrasound image sequence, the electronic device can delete the earliest obtained T pose sequences from the ultrasound probe pose sequence and calibration model pose sequence corresponding to the remaining ultrasound images based on the rate difference T between the ultrasound probe displacement signal and the ultrasound image displacement signal, and obtain the first transformation matrix and the second transformation matrix respectively.

[0079] In addition, the electronic device can calculate a corresponding three-dimensional (3D) marker sequence based on the calibration model and the 2D marker sequence obtained by the aforementioned 2D marker recognition, through similar triangles and coordinate transformation. The last T marker groups obtained in the 3D marker sequence should also be deleted, so that a third transformation matrix can be obtained. The 3D markers in each of the above marker groups belong to the same corresponding ultrasound image.

[0080] On this basis, the electronic device can calculate the fourth transformation matrix, that is, the spatial posture calibration matrix, based on the first transformation matrix, the second transformation matrix and the third transformation matrix to complete the spatial posture calibration of the ultrasound probe.

[0081] In a possible implementation of an embodiment of the present application, after obtaining the fourth transformation matrix, ie, the spatial posture transformation matrix, through the above calculation, the electronic device may further modify the spatial posture matrix.

[0082] In an embodiment of the present application, the electronic device can discard the noise data in the 3D marker point sequence, and then determine the target 3D marker point sequence based on the spatial pose calibration matrix and the 2D marker point sequence, which is also the theoretical 3D marker point sequence obtained by deduction. The error between the 3D marker point sequence after discarding the noise data and the above-mentioned target 3D marker point sequence is then calculated. The electronic device can correct the spatial pose calibration matrix according to the error. Specifically, if the above-mentioned error is greater than the set error range, the electronic device can discard the marker point with the largest error in the 3D marker point sequence. At the same time, the 2D marker point corresponding to the discarded 3D marker point will also be discarded. Then, the fourth transformation matrix can be recalculated, and the above operation is repeated until the error is within the set error range. In this way, the final corrected fourth transformation matrix is ​​used as the spatial pose calibration matrix to complete the spatial pose calibration of the ultrasound probe.

[0083] In another possible implementation of the embodiment of the present application, after completing the aforementioned time calibration and spatial posture calibration, the calibration results can be further optimized.

[0084] Because ultrasound slices are 2D images, the accuracy of the pose matrix is ​​particularly important for the calibration results in the spatial pose calibration matrix. Slight errors in the rotational component can make the calibration results unstable, which is also a common problem in actual use. The embodiments of the present application can stabilize the pose matrix through calibration optimization, thereby improving the stability of the calibration results.

[0085] In an embodiment of the present application, the fourth transformation matrix can be optimized using a modified Powell algorithm based on the above-mentioned 2D marker point sequence and 3D marker point sequence. The Powell algorithm is an algorithm for solving the local minimum of a function. Specifically, the search direction can be initialized first, the direction pool of the initial search can be set, the loss function f(x) can be iteratively calculated, and the search direction can be updated until the minimum value of f(x) meets the set threshold, or the iterative cycle ends, the search process is exited, and finally the optimal solution of the fourth transformation matrix can be obtained, which is the optimized fourth transformation matrix. The electronic device can use the optimized fourth transformation matrix as the posture calibration matrix of the ultrasonic probe. At this point, the calibration of the ultrasonic probe is completed.

[0086] In the embodiments of the present application, image recognition is used to reduce the error in identifying marker points from ultrasound images; by filtering and discarding noise data in the collected data, the accuracy of subsequent processing results is guaranteed; by combining time calibration and spatial posture calibration to calibrate the ultrasound probe, the problem of inconsistent data transmission rates between different ultrasound devices and optical navigation devices is solved, and it can ensure that the ultrasound image sequence required for calibration, the ultrasound probe posture sequence, and the calibration model posture sequence correspond one to one. Applying the calibration method provided in the embodiments of the present application can improve the calibration accuracy of the ultrasound probe and the stability of the calibration results.

[0087] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0088] For ease of understanding, the ultrasonic probe calibration method provided in the embodiment of the present application is introduced below with reference to a complete example.

[0089] like Figure 2 FIG. 1 is a schematic diagram of a calibration process of an ultrasonic probe provided in an embodiment of the present application. Figure 2The calibration process shown in the figure can be summarized into five parts: data acquisition, image recognition, time calibration, spatial pose calibration, and calibration optimization. The details are described below.

[0090] 1. Data Collection

[0091] For data acquisition, the electronic device can be connected to the optical navigation device and the ultrasound device. Tracers can be installed on the ultrasound probe and calibration phantom. For example, a first tracer can be installed on the ultrasound probe, and a second tracer can be installed on the calibration phantom. The calibration phantom can use the multi-layer N-wire phantom with a bottom surface made of acoustic wave reflective material, as described above.

[0092] The optical navigation device should be placed within the optimal acquisition range, the multi-threaded acquisition function of the electronic device should be started, and the ultrasound probe should be placed above the calibration model and slid. The electronic device can simultaneously acquire the ultrasound probe pose sequence, the calibration model pose sequence, and the ultrasound image sequence.

[0093] 2. Image Recognition

[0094] In the image recognition part, the electronic device can perform marker point recognition and line recognition based on the acquired two-dimensional ultrasound image sequence.

[0095] 2.1 2D landmark recognition

[0096] 2D marker recognition in the present application refers to identifying a sequence of 2D markers from a 2D ultrasound image sequence. The 2D markers are derived from line segments in the calibration model. Due to the nature of ultrasound imaging, each marker appears as a small, irregular region in the image. Therefore, image processing and analysis are required for each acquired 2D ultrasound image.

[0097] First, image preprocessing, such as histogram equalization and threshold segmentation, is required. Connected domain analysis is then performed to identify ROIs based on their characteristics. Selecting ROIs can improve algorithm speed. The area and centroid of each connected domain are calculated to obtain the average area. Connected domains are then screened based on the number of line segments in the calibration model to obtain multiple candidate connected domains.

[0098] According to the centroid of the candidate connected domain, with the horizontal direction as the search direction, a collinearity analysis is performed on the centroid to obtain the collinear connected domain. There are a total of M collinear connected domain groups (the number of which is an N-line model with M layers). Each collinear connected domain group has three connected domains. It is necessary to perform a three-point straight line fitting on the centroid of the three connected domains. The obtained fitting points can be used as candidate 2D marker points.

[0099] Perform a parallelism check on the candidate 2D marker point sequences and calculate the slope of each collinear connected domain group. If all slope values ​​are within a certain parallelism range, the parallelism check passes. If any slope value does not meet the parallelism range, the corresponding ultrasound image is marked as the first noise image.

[0100] After passing the parallelism check, a distance check is performed to calculate the distance between each group of collinear connected domains. If the distances of all lines are within a certain range, the distance check passes. If any distance does not meet the requirements, the corresponding ultrasound image is marked as the first noise image.

[0101] After the time calibration is completed, when the spatial pose calibration is required, the corresponding calibration model pose and ultrasound probe pose need to be discarded from the sequence according to the marked first noise image.

[0102] 2.2 Line Recognition

[0103] Similar to 2D marker recognition, the ultrasound image must first be preprocessed with image denoising and contrast enhancement. The image is then cropped horizontally. The resulting multiple cropped regions are analyzed for their maximum points, and the coordinates of all regional maximum points are recorded. A straight line is fitted to the resulting set of regional maximum points. If the slope of the resulting line is within a certain range, the image passes verification; otherwise, the image is marked as a second noise image. The resulting straight line is intersected with the vertical midline of the image to obtain an intersection point. Based on the image sequence, an intersection point sequence is obtained. Each intersection point in the intersection point sequence is normalized to the range (1, 0) based on the vertical direction of the image. This yields the displacement signal of the ultrasound image.

[0104] During time calibration, the corresponding ultrasound probe posture needs to be discarded from the sequence according to the marked second noise image.

[0105] 3. Time calibration

[0106] Time calibration addresses the issue of inconsistent transmission rates between ultrasound devices and optical navigation systems. This process provides the difference in rate between the two, providing a foundation for subsequent ultrasound probe position calibration. This difference in rate can be measured as the difference in rate between the ultrasound probe displacement signal and the ultrasound image displacement signal.

[0107] Specifically, the ultrasound probe position sequence can be extracted from the ultrasound probe pose sequence obtained by the optical navigation device. The covariance matrix of the position sequence is calculated to obtain the eigenvectors and eigenvalues. The eigenvector corresponding to the maximum eigenvalue is used as the principal direction of the position sequence. The projection of the position sequence along this principal direction is then calculated to obtain the projection sequence. The displacement signal of the ultrasound probe can be obtained by normalizing the projection points of the projection sequence. Because the image motion signal is opposite to the actual probe movement direction, the normalization range of the projection points is opposite to that of the intersection point sequence, and the normalization range of the projection points is (0, 1).

[0108] Based on the displacement signal of the ultrasound image and the displacement signal of the ultrasound probe, a line graph of the two displacement signals can be obtained. Since the position output rate of the optical navigation device is faster than the image output rate of the ultrasound device, the ultrasound probe displacement signal is moved to align the image displacement signal with the image displacement signal as the reference. The SSD is used as the loss assessment value. When the loss value is minimized, the rate difference between the two displacement signals is obtained, thus completing the time calibration.

[0109] 4. Spatial pose calibration

[0110] In the spatial pose calibration part, the 3D marker point sequence can be calculated based on the identified 2D marker point sequence and the calibration model through similar triangles and coordinate transformation.

[0111] Based on the time-calibrated rate difference T, the first transformation matrix is ​​obtained by deleting the T earliest acquired data points from the ultrasound probe pose sequence. The second transformation matrix is ​​obtained by deleting the T earliest acquired data points from the calibration model pose sequence. The third transformation matrix is ​​obtained by deleting the T latest acquired data points from the 2D and 3D marker point sequences. Based on the first, second, and third transformation matrices, the spatial pose calibration matrix of the ultrasound probe is obtained, which is the fourth transformation matrix.

[0112] On this basis, the fourth transformation matrix can also be corrected. Specifically, the noise data in the 3D marker point sequence can be discarded, and the derived 3D marker point sequence, namely the aforementioned target 3D marker point sequence, can be calculated using the fourth transformation matrix and the 2D marker point sequence. The 3D marker point sequence after discarding the noise data can be used to perform error calculation with the above-mentioned target 3D marker point sequence. For example, the CR error can be used to calculate the difference between the two sequences. If the CR error value is greater than a certain range, the marker point with the largest error in the 3D marker point sequence can be discarded, and the corresponding 2D marker point should also be discarded. At this time, the fourth transformation matrix needs to be recalculated until the CR error value meets the set threshold.

[0113] 5. Calibration optimization

[0114] Since ultrasound slices are 2D images, the accuracy of the attitude matrix in the spatial pose calibration matrix is ​​particularly important to the calibration results. A slight error in the rotation component will make the calibration results unstable, which is also a common problem in actual use. Calibration optimization can make the attitude matrix stable.

[0115] Based on the aforementioned 2D and 3D marker point sequences, the modified Powell method can be used to optimize the fourth transformation matrix. First, the search direction is initialized, and the initial search direction pool is set. The loss function f(x) is iteratively calculated and the search direction is updated until the minimum value of f(x) meets the set threshold or the iterative cycle ends. The search process is then exited, and the optimal solution for the fourth transformation matrix is ​​finally obtained.

[0116] At this point, the calibration of the ultrasound probe is completed.

[0117] In combination with the various embodiments described above, the present application also provides a calibration system for an ultrasound probe. Figure 3 FIG. 1 is a schematic diagram of a calibration system for an ultrasound probe provided in an embodiment of the present application. Figure 3 The calibration system 300 may include a calibration model 310, an electronic device 320, and an optical navigation device 330 and an ultrasound device 340 connected to the electronic device 320. The calibration model 310 is a multi-layer N-line model with a sound wave reflecting material on the bottom. The ultrasound device 340 includes an ultrasound probe 341. A first tracer and a second tracer are installed on the ultrasound probe 341 and the calibration model 310, respectively.

[0118] The optical navigation device 330 may be used to obtain an ultrasound probe pose sequence and a calibration model pose sequence by identifying the first tracer and the second tracer;

[0119] The ultrasound device 340 may be used to collect ultrasound images while the ultrasound probe 341 is placed on the calibration model 310 and slides;

[0120] The electronic device 320 can be used to implement each step in the aforementioned method embodiments. For example, the electronic device 320 can receive the ultrasound probe pose sequence and the calibration model pose sequence obtained by the optical navigation device 330 by identifying the first tracer and the second tracer; receive the ultrasound image sequence obtained by the ultrasound device 340 by acquiring ultrasound images; determine the first noise image and the second noise image in the ultrasound image sequence by performing marker point recognition and straight line recognition on the ultrasound image sequence; after deleting the second noise image from the ultrasound image sequence, perform time calibration on the ultrasound probe 341 to obtain the rate difference between the ultrasound probe displacement signal and the ultrasound image displacement signal; after deleting the first noise image from the ultrasound image sequence, perform spatial pose calibration on the ultrasound probe 341 based on the rate difference to obtain the spatial pose calibration matrix of the ultrasound probe 341, thereby completing the calibration of the ultrasound probe 341.

[0121] for Figure 3 The various components of the calibration system 300 and the functions that can be achieved can be found in the introduction of the aforementioned method embodiment part, and will not be repeated here.

[0122] Reference Figure 4 , shows a schematic diagram of an ultrasound probe calibration device provided in an embodiment of the present application, which may specifically include a pose sequence receiving module 401, an image sequence receiving module 402, a noise image recognition module 403, a time calibration module 404, and a space calibration module 405, wherein:

[0123] A pose sequence receiving module 401 is configured to receive an ultrasound probe pose sequence and a calibration model pose sequence obtained by the optical navigation device by identifying a first tracer and a second tracer, wherein the first tracer is mounted on the ultrasound probe, and the second tracer is mounted on the calibration model, wherein the calibration model is a multi-layer N-line model having a sound wave reflecting material on the bottom.

[0124] An image sequence receiving module 402 is configured to receive an ultrasound image sequence obtained by an ultrasound device through acquisition of ultrasound images, wherein the ultrasound images are acquired by sliding the ultrasound probe over the calibration model;

[0125] a noise image recognition module 403 configured to determine a first noise image and a second noise image in the ultrasound image sequence by performing marker point recognition and line recognition on the ultrasound image sequence, wherein the first noise image is an image in the ultrasound image sequence that does not meet the marker point recognition requirement, and the second noise image is an image in the ultrasound image sequence that does not meet the line recognition requirement;

[0126] a time calibration module 404 configured to perform time calibration on the ultrasound probe after deleting the second noise image from the ultrasound image sequence to obtain a rate difference between the ultrasound probe displacement signal and the ultrasound image displacement signal;

[0127] The spatial calibration module 405 is configured to perform spatial pose calibration on the ultrasound probe based on the velocity difference after deleting the first noise image from the ultrasound image sequence, to obtain a spatial pose calibration matrix of the ultrasound probe.

[0128] In the embodiment of the present application, the marker points in the ultrasound image sequence are used to represent the line segments in the calibration model, and the noise image recognition module 403 can be specifically used to:

[0129] performing a connected domain analysis on each ultrasound image in the ultrasound image sequence based on the number of line segments in the calibration model to obtain a plurality of collinear connected domain groups, and determining a plurality of candidate marker points based on the plurality of collinear connected domain groups;

[0130] By performing a first verification on the plurality of candidate marker points, the ultrasound images corresponding to the candidate marker points that do not meet the first verification requirement are identified as first noise images;

[0131] Cropping each ultrasound image in the ultrasound image sequence in a horizontal direction to obtain a plurality of cropped regions of each ultrasound image, and determining a regional maximum point corresponding to a pixel maximum value in each cropped region of each ultrasound image;

[0132] A second verification is performed on the corresponding ultrasound image based on the maximum point of the region in each of the cropped regions, and the ultrasound image that does not meet the second verification requirement is identified as a second noise image.

[0133] In a possible implementation of the embodiment of the present application, the noise image recognition module 403 may also be used to:

[0134] Identifying multiple connected domains contained in each ultrasound image in the ultrasound image sequence, and obtaining the same number of multiple candidate connected domains based on the number of line segments in the calibration model;

[0135] Performing collinearity analysis on the plurality of candidate connected domains to obtain a plurality of collinear connected domain groups, wherein the number of the plurality of collinear connected domain groups is the same as the number of layers of the calibration model;

[0136] Linear fitting is performed on the centroids of the connected domains in each of the collinear connected domain groups, and the obtained fitting points are determined as candidate marking points.

[0137] In another possible implementation of the embodiment of the present application, the first verification includes a parallelism verification and a distance verification, and the noise image recognition module 403 may also be used to:

[0138] respectively calculating first slopes of first straight lines obtained by fitting the plurality of candidate marker points in each of the collinear connected domain groups; if the first slopes are within a preset first slope range, determining that the ultrasound images corresponding to the plurality of candidate marker points in the collinear connected domain group pass the parallelism check;

[0139] respectively calculating the distance between each of the collinear connected domain groups; if the distance is within a preset distance range, determining that the ultrasound images corresponding to the plurality of candidate marker points in the collinear connected domain group pass the distance check;

[0140] An ultrasound image that fails the parallelism check or the distance check is identified as a first noise image.

[0141] In another possible implementation of the embodiment of the present application, the noise image recognition module 403 may also be used to:

[0142] calculating a second slope of a second straight line obtained by fitting based on the regional maximum point in each of the cropped regions; if the second slope is within a preset second slope range, determining that the corresponding ultrasound image passes the second verification;

[0143] An ultrasound image that fails the second verification is identified as a second noise image.

[0144] In another possible implementation of the embodiment of the present application, the noise image recognition module 403 may also be used to:

[0145] For the ultrasound images that pass the second verification, determining the intersection of the second straight line obtained by fitting in each ultrasound image and the vertical midline of the current ultrasound image, to obtain a sequence of intersections corresponding to the multiple ultrasound images;

[0146] Taking the vertical direction of the image as a reference, normalization processing is performed on each intersection point in the intersection point sequence to obtain an ultrasonic image displacement signal.

[0147] In the embodiment of the present application, the time calibration module 404 may be specifically used to:

[0148] After deleting the second noise image from the ultrasound image sequence, extracting an ultrasound probe position sequence from an ultrasound probe pose sequence corresponding to the remaining ultrasound images;

[0149] By calculating the covariance matrix of the position sequence, a plurality of eigenvectors and corresponding plurality of eigenvalues ​​are obtained, and the eigenvector corresponding to the maximum eigenvalue is used as the main direction of the ultrasound probe position sequence;

[0150] Calculating the projection of the ultrasound probe position sequence in the main direction to obtain a projection sequence, and performing normalization processing on the projection sequence to obtain an ultrasound probe displacement signal;

[0151] A velocity difference between the ultrasound probe displacement signal and the ultrasound image displacement signal is calculated.

[0152] In a possible implementation of the embodiment of the present application, the time calibration module 404 may also be used to:

[0153] moving the ultrasonic probe displacement signal based on the ultrasonic image displacement signal to align the ultrasonic probe displacement signal with the ultrasonic image displacement signal;

[0154] Calculating the loss value of the aligned ultrasonic probe displacement signal and the ultrasonic image displacement signal using a target detection algorithm SSD;

[0155] A rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal corresponding to when the loss value is minimum is determined.

[0156] In the embodiment of the present application, the spatial pose calibration module 405 can be specifically used to:

[0157] After deleting the first noise image from the ultrasound image sequence, deleting the earliest T pose sequences from the ultrasound probe pose sequence and the calibration model pose sequence corresponding to the remaining ultrasound images, to obtain a first transformation matrix and a second transformation matrix, respectively; wherein T is equal to the rate difference;

[0158] Determining a three-dimensional marker point sequence based on the calibration model and the two-dimensional marker point sequence obtained by the marker point recognition, and deleting the latest T marker point groups obtained from the three-dimensional marker point sequence to obtain a third transformation matrix, wherein the three-dimensional marker points in each marker point group belong to the same corresponding ultrasound image;

[0159] The spatial pose calibration matrix is ​​calculated based on the first transformation matrix, the second transformation matrix and the third transformation matrix.

[0160] In a possible implementation of the embodiment of the present application, the spatial pose calibration module 405 may also be used to:

[0161] Determine a target three-dimensional marker point sequence based on the spatial pose calibration matrix and the two-dimensional marker point sequence;

[0162] After discarding noise data from the three-dimensional marker point sequence, calculating an error between the three-dimensional marker point sequence after discarding the noise data and the target three-dimensional marker point sequence;

[0163] The spatial pose calibration matrix is ​​corrected according to the error.

[0164] An embodiment of the present application provides a calibration device for an ultrasound probe, and by using this device, each step in the aforementioned method embodiments can be implemented.

[0165] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.

[0166] Reference Figure 5 , shows a schematic diagram of an electronic device provided by an embodiment of the present application. Figure 5 As shown, the electronic device 500 in the embodiment of the present application includes: a processor 510, a memory 520, and a computer program 521 stored in the memory 520 and executable on the processor 510. When the processor 510 executes the computer program 521, the steps of each embodiment of the above-mentioned ultrasound probe calibration method are implemented, such as Figure 1 Alternatively, when the processor 510 executes the computer program 521, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 4 Functions of modules 401 to 405 are shown.

[0167] Exemplarily, the computer program 521 can be divided into one or more modules / units, which are stored in the memory 520 and executed by the processor 510 to complete the present application. The one or more modules / units can be a series of computer program instruction segments that can perform specific functions, and the instruction segments can be used to describe the execution process of the computer program 521 in the electronic device 500. For example, the computer program 521 can be divided into a pose sequence receiving module, an image sequence receiving module, a noise image recognition module, a time calibration module, and a space calibration module. The specific functions of each module are as follows:

[0168] a pose sequence receiving module, configured to receive an ultrasound probe pose sequence and a calibration model pose sequence obtained by the optical navigation device by identifying a first tracer and a second tracer, wherein the first tracer is mounted on the ultrasound probe, and the second tracer is mounted on the calibration model, wherein the calibration model is a multi-layer N-line model having a sound wave reflecting material on the bottom;

[0169] An image sequence receiving module, configured to receive an ultrasound image sequence obtained by an ultrasound device through acquisition of ultrasound images, wherein the ultrasound images are acquired by placing the ultrasound probe on the calibration model and sliding the probe;

[0170] a noise image recognition module, configured to determine a first noise image and a second noise image in the ultrasound image sequence by performing marker point recognition and line recognition on the ultrasound image sequence, wherein the first noise image is an image in the ultrasound image sequence that does not meet the marker point recognition requirement, and the second noise image is an image in the ultrasound image sequence that does not meet the line recognition requirement;

[0171] a time calibration module, configured to perform time calibration on the ultrasound probe after deleting the second noise image from the ultrasound image sequence, to obtain a rate difference between the ultrasound probe displacement signal and the ultrasound image displacement signal;

[0172] A spatial calibration module is used to perform spatial posture calibration on the ultrasound probe based on the rate difference after deleting the first noise image from the ultrasound image sequence, so as to obtain a spatial posture calibration matrix of the ultrasound probe.

[0173] The electronic device 500 may be a device capable of implementing the functions of each step in the aforementioned method embodiments, and may be a desktop computer, a cloud server, or other device. The electronic device 500 may include, but is not limited to, a processor 510 and a memory 520. Those skilled in the art will appreciate that Figure 5 It is only an example of the electronic device 500 and does not constitute a limitation of the electronic device 500. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 500 may also include input and output devices, network access devices, buses, etc.

[0174] The processor 510 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0175] The memory 520 can be an internal storage unit of the electronic device 500, such as a hard drive or memory of the electronic device 500. The memory 520 can also be an external storage device of the electronic device 500, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the electronic device 500. Furthermore, the memory 520 can include both an internal storage unit of the electronic device 500 and an external storage device. The memory 520 is used to store the computer program 521 and other programs and data required by the electronic device 500. The memory 520 can also be used to temporarily store data that has been output or is about to be output.

[0176] An embodiment of the present application further discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the methods described in the aforementioned embodiments are implemented.

[0177] An embodiment of the present application further discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a computer, the methods described in the aforementioned embodiments are implemented.

[0178] An embodiment of the present application further discloses a computer program product, including a computer program. When the computer program is run on a computer, the computer is caused to execute the methods described in the aforementioned embodiments.

[0179] The above embodiments are intended only to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the above embodiments or replace some of the technical features therein with equivalents; and such modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application and should be included within the scope of protection of the present application.

Claims

1. A method for calibrating an ultrasonic probe, characterized in that: include: receiving an ultrasound probe pose sequence and a calibration model pose sequence obtained by an optical navigation device by identifying a first tracer and a second tracer, wherein the first tracer is mounted on the ultrasound probe, and the second tracer is mounted on the calibration model, wherein the calibration model is a multi-layer N-line model with a sound wave reflecting material on the bottom; receiving an ultrasound image sequence obtained by an ultrasound device through acquisition of ultrasound images, wherein the ultrasound images are acquired by placing the ultrasound probe on the calibration model and sliding it; Determining a first noise image and a second noise image in the ultrasound image sequence by performing marker point recognition and line recognition on the ultrasound image sequence, wherein the first noise image is an image in the ultrasound image sequence that does not meet the marker point recognition requirement, and the second noise image is an image in the ultrasound image sequence that does not meet the line recognition requirement; After deleting the second noise image from the ultrasonic image sequence, performing time calibration on the ultrasonic probe to obtain a rate difference between an ultrasonic probe displacement signal and an ultrasonic image displacement signal; After deleting the first noise image from the ultrasound image sequence, the ultrasound probe is spatially calibrated based on the rate difference to obtain a spatial pose calibration matrix of the ultrasound probe.

2. The method according to claim 1, characterized in that The marker points in the ultrasound image sequence are used to represent line segments in the calibration model, and determining the first noise image and the second noise image in the ultrasound image sequence by performing marker point recognition and line recognition on the ultrasound image sequence includes: performing a connected domain analysis on each ultrasound image in the ultrasound image sequence based on the number of line segments in the calibration model to obtain a plurality of collinear connected domain groups, and determining a plurality of candidate marker points based on the plurality of collinear connected domain groups; By performing a first verification on the plurality of candidate marker points, the ultrasound images corresponding to the candidate marker points that do not meet the first verification requirement are identified as first noise images; Cropping each ultrasound image in the ultrasound image sequence in a horizontal direction to obtain a plurality of cropped regions of each ultrasound image, and determining a regional maximum point corresponding to a pixel maximum value in each cropped region of each ultrasound image; A second verification is performed on the corresponding ultrasound image based on the maximum point of the region in each of the cropped regions, and the ultrasound image that does not meet the second verification requirement is identified as a second noise image.

3. The method according to claim 2, characterized in that The method further comprises: performing a connected domain analysis on each ultrasound image in the ultrasound image sequence based on the number of line segments in the calibration model to obtain a plurality of collinear connected domain groups, and determining a plurality of candidate marker points based on the plurality of collinear connected domain groups, including: Identifying multiple connected domains contained in each ultrasound image in the ultrasound image sequence, and obtaining the same number of multiple candidate connected domains based on the number of line segments in the calibration model; Performing collinearity analysis on the plurality of candidate connected domains to obtain a plurality of collinear connected domain groups, wherein the number of the plurality of collinear connected domain groups is the same as the number of layers of the calibration model; Linear fitting is performed on the centroids of the connected domains in each of the collinear connected domain groups, and the obtained fitting points are determined as candidate marking points.

4. The method according to claim 2, characterized in that The first verification includes a parallelism verification and a distance verification. The first verification is performed on the plurality of candidate marker points, and the ultrasound images corresponding to the candidate marker points that do not meet the first verification requirements are identified as first noise images, including: respectively calculating first slopes of first straight lines obtained by fitting the plurality of candidate marker points in each of the collinear connected domain groups; if the first slopes are within a preset first slope range, determining that the ultrasound images corresponding to the plurality of candidate marker points in the collinear connected domain group pass the parallelism check; respectively calculating the distance between each of the collinear connected domain groups; if the distance is within a preset distance range, determining that the ultrasound images corresponding to the plurality of candidate marker points in the collinear connected domain group pass the distance check; An ultrasound image that fails the parallelism check or the distance check is identified as a first noise image.

5. The method according to any one of claims 2 to 4, characterized in that The performing a second verification on the corresponding ultrasound image based on the regional maximum point in each of the cropped regions, and identifying the ultrasound image that does not meet the second verification requirement as a second noise image, includes: calculating a second slope of a second straight line obtained by fitting based on the regional maximum point in each of the cropped regions; if the second slope is within a preset second slope range, determining that the corresponding ultrasound image passes the second verification; An ultrasound image that fails the second verification is identified as a second noise image.

6. The method according to claim 5, characterized in that Also includes: For the ultrasound images that pass the second verification, determining the intersection of the second straight line obtained by fitting in each ultrasound image and the vertical midline of the current ultrasound image, to obtain a sequence of intersections corresponding to the multiple ultrasound images; Taking the vertical direction of the image as a reference, normalization processing is performed on each intersection point in the intersection point sequence to obtain an ultrasonic image displacement signal.

7. The method according to any one of claims 1 to 4 or 6, characterized in that After deleting the second noise image from the ultrasonic image sequence, performing time calibration on the ultrasonic probe to obtain a rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal includes: After deleting the second noise image from the ultrasound image sequence, extracting an ultrasound probe position sequence from an ultrasound probe pose sequence corresponding to the remaining ultrasound images; By calculating the covariance matrix of the position sequence, a plurality of eigenvectors and corresponding plurality of eigenvalues ​​are obtained, and the eigenvector corresponding to the maximum eigenvalue is used as the main direction of the ultrasound probe position sequence; Calculating the projection of the ultrasound probe position sequence in the main direction to obtain a projection sequence, and performing normalization processing on the projection points of the projection sequence to obtain an ultrasound probe displacement signal; A velocity difference between the ultrasound probe displacement signal and the ultrasound image displacement signal is calculated.

8. The method according to claim 7, characterized in that The calculating the rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal includes: moving the ultrasonic probe displacement signal based on the ultrasonic image displacement signal to align the ultrasonic probe displacement signal with the ultrasonic image displacement signal; Calculating the loss value of the aligned ultrasonic probe displacement signal and the ultrasonic image displacement signal using a target detection algorithm SSD; A rate difference between the ultrasonic probe displacement signal and the ultrasonic image displacement signal corresponding to when the loss value is minimum is determined.

9. The method according to any one of claims 1 to 4 or 6 or 8, characterized in that After deleting the first noise image from the ultrasound image sequence, performing spatial pose calibration on the ultrasound probe based on the rate difference to obtain a spatial pose calibration matrix of the ultrasound probe includes: After deleting the first noise image from the ultrasound image sequence, deleting the earliest T pose sequences from the ultrasound probe pose sequence and the calibration model pose sequence corresponding to the remaining ultrasound images, to obtain a first transformation matrix and a second transformation matrix, respectively; wherein the value of T is equal to the rate difference; Determining a three-dimensional marker point sequence based on the calibration model and the two-dimensional marker point sequence obtained by the marker point recognition, and deleting the latest T marker point groups obtained from the three-dimensional marker point sequence to obtain a third transformation matrix, wherein the three-dimensional marker points in each marker point group belong to the same corresponding ultrasound image; The spatial pose calibration matrix is ​​calculated based on the first transformation matrix, the second transformation matrix and the third transformation matrix.

10. The method according to claim 9, characterized in that Also includes: Determine a target three-dimensional marker point sequence based on the spatial pose calibration matrix and the two-dimensional marker point sequence; After discarding noise data from the three-dimensional marker point sequence, calculating an error between the three-dimensional marker point sequence after discarding the noise data and the target three-dimensional marker point sequence; The spatial pose calibration matrix is ​​corrected according to the error.

11. A calibration system for an ultrasonic probe, characterized in that: The system comprises a calibration model, an electronic device, and an optical navigation device and an ultrasonic device connected to the electronic device. The calibration model is a multi-layer N-line model with a sound wave reflecting material on the bottom. The ultrasonic device comprises an ultrasonic probe. A first tracer and a second tracer are mounted on the ultrasonic probe and the calibration model, respectively. The optical navigation device is used to obtain an ultrasound probe pose sequence and a calibration model pose sequence by identifying the first tracer and the second tracer; The ultrasound device is used to collect ultrasound images during the process of placing the ultrasound probe on the calibration model and sliding it; The electronic device is used to implement the method according to any one of claims 1 to 10.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the electronic device is caused to implement the method according to any one of claims 1 to 10.

Images