Ultrasonic three-dimensional model registration method, device, equipment and computer program product

Abstract

The invention is suitable for the technical field of medical treatment, and provides an ultrasonic three-dimensional model registration method, device and equipment and a computer program product. The ultrasonic three-dimensional model registration method comprises the following steps: acquiring a to-be-registered ultrasonic image and a to-be-registered three-dimensional model; registration parameters between the to-be-registered ultrasonic image and the to-be-registered three-dimensional model are determined, the registration parameters comprise scale parameters, a rotation matrix and a translation vector of the to-be-registered three-dimensional model, and the scale parameters are used for scaling the size of the to-be-registered three-dimensional model; and according to the registration parameters, registering the to-be-registered three-dimensional model to the to-be-registered ultrasonic image, and determining a deformed three-dimensional model. According to the embodiment of the invention, the three-dimensional model can be registered to the two-dimensional ultrasonic image, and the used registration parameter is added with consideration of the scale, so that the registration result is more accurate.

Description

Technical Field

[0001] The present application belongs to the field of medical technology, and in particular, relates to a method, device, equipment and computer program product for ultrasound three-dimensional model registration. Background Art

[0002] Ultrasound (US) images are widely used for large-scale screening and routine examinations, interventional treatments, and intraoperative navigation due to their real-time nature. However, the image quality of two-dimensional ultrasound images is limited, and there are only slice data, which brings a great operational burden to the operator. At present, one of the research hotspots is to accurately align three-dimensional models with two-dimensional ultrasound images, providing three-dimensional visualization functions for two-dimensional ultrasound images, which helps surgeons to more accurately grasp the patient's anatomical structure, thereby improving the accuracy and safety of surgery. However, in actual applications, it is found that since many three-dimensional model data do not come from the same patient as the two-dimensional ultrasound image, the registration scheme of the related technology has certain errors. Summary of the invention

[0003] The embodiments of the present application provide a method, apparatus, device and computer program product for ultrasound three-dimensional model registration, which can register a three-dimensional model to a two-dimensional ultrasound image, and the registration parameters used take scale into consideration, which can make the registration result more accurate.

[0004] A first aspect of an embodiment of the present application provides a method for ultrasound three-dimensional model registration, including: acquiring an ultrasound image to be registered and a three-dimensional model to be registered; determining registration parameters between the ultrasound image to be registered and the three-dimensional model to be registered, the registration parameters including scale parameters, a rotation matrix and a translation vector of the three-dimensional model to be registered, and the scale parameters are used to scale the size of the three-dimensional model to be registered; and according to the registration parameters, registering the three-dimensional model to be registered to the ultrasound image to be registered.

[0005] In some embodiments of the first aspect, determining the registration parameters between the ultrasound image to be registered and the three-dimensional model to be registered includes: determining matching point pairs, each of the matching point pairs including a contour point in the ultrasound image to be registered and a grid point on the three-dimensional model to be registered that matches the contour point; using a transformation parameter that minimizes the sum of each first distance as the registration parameter, each of the first distances being the distance between the grid point and the contour point transformed by the transformation parameter in one of the matching point pairs.

[0006] In some embodiments of the first aspect, the determining of matching point pairs includes: associating the ultrasound images to be registered in different phases with a three-dimensional model to be registered respectively; matching the contour points in the ultrasound images to be registered in each phase with the grid points on the associated three-dimensional model to be registered to obtain the matching point pairs.

[0007] In some embodiments of the first aspect, associating the ultrasound images to be registered at different phases with a three-dimensional model to be registered respectively includes: if there are three-dimensional models to be registered at different phases, then associating the ultrasound images to be registered with the three-dimensional models to be registered according to the phase of each ultrasound image to be registered and the phase of each three-dimensional model to be registered.

[0008] In some embodiments of the first aspect, associating the ultrasound images to be registered in different phases with a three-dimensional model to be registered respectively includes: if there is only a three-dimensional model to be registered in a single phase, associating all the ultrasound images to be registered with the same three-dimensional model to be registered.

[0009] In some embodiments of the first aspect, the transformation parameter that will minimize the sum of each first distance is used as the alignment parameter, including: correcting each first distance using a robustness loss function; and using the transformation parameter that will minimize the sum of each corrected first distance as the alignment parameter.

[0010] In some embodiments of the first aspect, after determining the deformed three-dimensional model, the method further includes: displaying the deformed three-dimensional model; and / or performing surgical navigation based on the deformed three-dimensional model.

[0011] A second aspect of an embodiment of the present application provides an apparatus for ultrasonic three-dimensional model registration, comprising: an acquisition unit, used to acquire an ultrasonic image to be registered and a three-dimensional model to be registered; a parameter determination unit, used to determine registration parameters between the ultrasonic image to be registered and the three-dimensional model to be registered, the registration parameters comprising a scale parameter, a rotation matrix and a translation vector of the three-dimensional model to be registered, and the scale parameter is used to scale the size of the three-dimensional model to be registered; a registration unit, used to align the three-dimensional model to be registered to the ultrasonic image to be registered according to the registration parameters, and determine the deformed three-dimensional model.

[0012] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method for ultrasound three-dimensional model registration as described in any one of the first aspects are implemented.

[0013] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned ultrasound three-dimensional model registration method are implemented.

[0014] A fifth aspect of an embodiment of the present application provides a computer program product, including a computer program. When the computer program is run, the method for ultrasound three-dimensional model registration described in the first aspect is executed.

[0015] In an embodiment of the present application, an ultrasound image to be registered and a three-dimensional model to be registered are obtained, and according to the registration parameters between the ultrasound image to be registered and the three-dimensional model to be registered, the three-dimensional model to be registered is registered with the ultrasound image to be registered, and the deformed three-dimensional model is determined, so that the deformed three-dimensional model is used to present the shape of the scanned object in the medical image to be registered. While the three-dimensional model is registered with the two-dimensional ultrasound image, since the registration parameters used have scale considerations, during the deformation process of the three-dimensional model to be registered to the deformed three-dimensional model, the size can be scaled according to the shape of the scanned object in the medical image to be registered, which can reduce the error caused by the three-dimensional model and the two-dimensional ultrasound image coming from different patients, and make the registration result more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0017] Figure 1 It is a schematic diagram of the implementation process of the method for ultrasound three-dimensional model registration provided in an embodiment of the present application;

[0018] Figure 2 It is a schematic diagram of a specific implementation process of determining the registration parameters provided in an embodiment of the present application;

[0019] Figure 3 is a schematic diagram of a comparison image provided in an embodiment of the present application;

[0020] Figure 4 It is a structural schematic diagram of a device for ultrasound three-dimensional model registration provided in an embodiment of the present application;

[0021] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are protected by the present application.

[0023] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0024] In the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0025] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0026] Accurately registering the 3D model with the 2D ultrasound image and providing 3D visualization for the 2D ultrasound image will help surgeons more accurately understand the patient's anatomical structure, thereby improving the accuracy and safety of the surgery. However, in actual applications, it is found that since many 3D model data do not come from the same patient as the 2D ultrasound image, the registration scheme of the related technology has certain errors.

[0027] In view of this, the present application proposes a method for ultrasound three-dimensional model registration, which can register the three-dimensional model to the two-dimensional ultrasound image, and the registration parameters used have scale considerations added to make the registration result more accurate.

[0028] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0029] Please refer to Figure 1 , Figure 1A schematic diagram of the implementation process of a method for ultrasound three-dimensional model alignment provided in an embodiment of the present application is shown. The method can be applied to an electronic device, which may be a computer, a smart phone, a medical device specifically used for image processing, or other smart devices, and the present application does not impose any restrictions on this.

[0030] Specifically, the above ultrasound three-dimensional model registration method may include the following steps S101 to S103.

[0031] Step S101, obtaining an ultrasound image to be registered and a three-dimensional model to be registered.

[0032] The ultrasound image to be registered and the three-dimensional model to be registered are a two-dimensional ultrasound image and a three-dimensional model that need to be registered with each other.

[0033] In some embodiments of the present application, the ultrasound image to be registered may be obtained by real-time scanning of a scanned object, and may reflect the real-time morphology of the scanned object. The scanned object may refer to a tissue, organ, or other object that needs to be imaged of a human or animal.

[0034] In some specific scenarios, real-time ultrasound images can be obtained by performing ultrasound scanning on the patient during surgery.

[0035] The three-dimensional model to be registered can be a preoperative three-dimensional model of the same patient as the ultrasound image to be registered, or a standardized three-dimensional model of the scanned object, and this application does not impose any restrictions on this. For example, when the scanned object is the heart, the three-dimensional model to be registered can be a three-dimensional model of the heart obtained by scanning the patient before surgery, or a standard three-dimensional model of the heart used uniformly by adults. It should be noted that the three-dimensional model to be registered can be a three-dimensional model obtained from non-ultrasound images.

[0036] The present application does not limit the method for obtaining the above-mentioned three-dimensional model to be registered. For example, it can be obtained through magnetic resonance imaging (MRI), computed tomography (CT), etc.

[0037] Step S102: determining the registration parameters between the ultrasound image to be registered and the three-dimensional model to be registered.

[0038] The registration parameters may include the scale parameter, rotation matrix and translation vector of the 3D model to be registered. The scale parameter is used to scale the size of the 3D model to be registered. The rotation matrix and translation vector are used to rotate and translate the 3D model to be registered.

[0039] Specifically, the registration parameters can be expressed as:

[0040] Here, R∈SO(3) is the rotation matrix, and SO(3) refers to a special orthogonal group containing the rotation matrix R, which can be called a three-dimensional rotation group. t∈R 3 is the translation vector, s∈R 1 is the scale parameter, Represents a 1*3 zero matrix. S is a 4×4 homogeneous form with a total of 7 degrees of freedom (i.e., 3 degrees of freedom for the rotation matrix, 3 degrees of freedom for the translation vector, and 1 degree of freedom for the scale parameter).

[0041] Step S103: register the three-dimensional model to be registered with the ultrasound image to be registered according to the registration parameters, and determine the deformed three-dimensional model.

[0042] In the implementation manner of the present application, there is often a certain difference between the morphology of the scanned object presented by the three-dimensional model to be registered and the morphology of the scanned object in the ultrasound image to be registered. In order to reduce or even avoid such a difference, the three-dimensional model to be registered can be registered with the ultrasound image to be registered according to the registration parameters to determine the deformed three-dimensional model. The deformed three-dimensional model is used to present the morphology of the scanned object in the medical image to be registered. In other words, the three-dimensional model to be registered can be registered with the medical image to be registered, so that the morphology of the scanned object presented by the deformed three-dimensional model is consistent with the morphology presented by the scanned object in the medical image to be registered.

[0043] It is understandable that if the ultrasound image to be registered and the three-dimensional model to be registered are from different patients, for example, the ultrasound image to be registered is an ultrasound image of a child, and the three-dimensional model to be registered is a three-dimensional model of an adult, then there is a large difference in the scale between the ultrasound image to be registered and the three-dimensional model to be registered. Even if the ultrasound image to be registered and the three-dimensional model to be registered are from the same patient, the scale between the ultrasound image to be registered and the three-dimensional model to be registered is likely to be different based on the different scanning times or the different physiological states of the patients during the scanning. In an embodiment of the present application, the registration parameters may include the scale parameters of the three-dimensional model to be registered, and then when the three-dimensional model to be registered is registered with the ultrasound image to be registered, in addition to rotation and translation, the size of the three-dimensional model to be registered can also be scaled so that the deformed three-dimensional model and the ultrasound image to be registered are consistent in scale.

[0044] In an embodiment of the present application, an ultrasound image to be registered and a three-dimensional model to be registered are obtained, and according to the registration parameters between the ultrasound image to be registered and the three-dimensional model to be registered, the three-dimensional model to be registered is registered with the ultrasound image to be registered, and the deformed three-dimensional model is determined, so that the deformed three-dimensional model is used to present the shape of the scanned object in the medical image to be registered. While the three-dimensional model is registered with the two-dimensional ultrasound image, since the registration parameters used have scale considerations, during the deformation process of the three-dimensional model to be registered to the deformed three-dimensional model, the size can be scaled according to the shape of the scanned object in the medical image to be registered, which can reduce the error caused by the three-dimensional model and the two-dimensional ultrasound image coming from different patients, and make the registration result more accurate.

[0045] In some embodiments of the present application, Figure 2 As shown, determining the registration parameters between the ultrasound image to be registered and the three-dimensional model to be registered may include: step S201 to step S202.

[0046] Step S201, determining matching point pairs.

[0047] Each matching point pair includes a contour point in the ultrasound image to be registered and a grid point on the three-dimensional model to be registered that matches the contour point.

[0048] Specifically, contour points can be obtained by performing image segmentation on the ultrasound image. Image segmentation can be implemented using a deep learning network or other existing image segmentation algorithms, which is not limited in this application. For each contour point on the ultrasound image to be registered, the nearest three-dimensional grid point can be selected on the three-dimensional model to be registered as a matching grid point to form a matching point pair. The distance here can be calculated using the Euclidean distance.

[0049] Step S202: using the transformation parameter that minimizes the sum of the first distances as the registration parameter.

[0050] Each first distance is the distance between a grid point and a contour point transformed by the transformation parameter in a matching point pair. Specifically, by transforming the contour point using the transformation parameter, the transformed coordinates of the contour point can be obtained, and the distance between the transformed coordinates and the coordinates of the grid point that forms a matching point pair with the contour point can be calculated to obtain the first distance of the matching point pair. The first distances of different matching point pairs can be accumulated to obtain the sum of the first distances.

[0051] The transformation parameter that minimizes the sum of the first distances means that each pair of matching points can be aligned as much as possible using the transformation parameter. Therefore, using the transformation parameter that minimizes the sum of the first distances as the registration parameter can make the registration parameter more accurate.

[0052] In some embodiments of the present application, using a transformation parameter that minimizes the sum of each first distance as a registration parameter may include: correcting each first distance using a robustness loss function; and using a transformation parameter that minimizes the sum of each corrected first distance as a registration parameter.

[0053] Among them, the robust loss function is used to reduce abnormal noise in the ultrasound image. The absolute value loss function, Huber loss function or other existing robust loss functions can be used, and this application does not impose any restrictions on this.

[0054] In some embodiments of the present application, determining matching point pairs may include: associating ultrasound images to be registered in different phases with a three-dimensional model to be registered respectively; matching the contour points in the ultrasound image to be registered in each phase with the grid points on the associated three-dimensional model to be registered to obtain matching point pairs.

[0055] Specifically, the present application can register a group of ultrasound image sequences, which may include multiple ultrasound images to be registered, and each ultrasound image to be registered may correspond to a phase. Among them, the phase is to divide the inspection process into different time periods according to different physiological or pathological processes. For example, when the scanned object is the heart, according to the cardiac cycle, the phase may include but is not limited to the systolic phase and the diastolic phase. When the scanned object is the liver, the phase may include but is not limited to the portal vein phase and the arterial phase.

[0056] The ultrasound images to be registered in different phases are respectively associated with a three-dimensional model to be registered. When determining the matching grid points for the contour points, the grid points can be determined in the associated three-dimensional model to be registered.

[0057] Specifically, in some implementations of the present application, determining the registration parameters can be divided into two modes, namely a fixed mode and a dynamic mode.

[0058] The two modes are described below.

[0059] Fixed mode:

[0060] In this mode, there is only a single phase of the three-dimensional model to be registered, for example, there is only one three-dimensional model for registration.

[0061] At this time, the ultrasound images to be registered in different phases are respectively associated with a three-dimensional model to be registered, including: if there is only a three-dimensional model to be registered in a single phase, all the ultrasound images to be registered are associated with the same three-dimensional model to be registered.

[0062] Accordingly, the objective function can be constructed as follows:

[0063]

[0064] in, is the jth contour point segmented from the ultrasound image obtained by the kth scan, p i is with The matching grid points, Ω is the set of grid points, S is the transformation parameter, and ρ is the robustness loss function.

[0065] The objective function (1) means to find the minimum value of the sum of the first distances. By optimizing the objective function (1), the transformation parameter S that minimizes the sum of the first distances can be used as the registration parameter S. * .

[0066] Dynamic Mode:

[0067] In this mode, there are multiple phases of 3D models to be registered.

[0068] At this time, the ultrasound images to be registered at different phases are respectively associated with a three-dimensional model to be registered, including: if there are three-dimensional models to be registered at different phases, the ultrasound images to be registered and the three-dimensional models to be registered are associated according to the phase of each ultrasound image to be registered and the phase of each three-dimensional model to be registered.

[0069] Specifically, each ultrasound image to be registered may be associated with a three-dimensional model to be registered in the same phase or a three-dimensional model to be registered in the closest phase.

[0070] Accordingly, the objective function can be constructed as follows:

[0071]

[0072] Where m is the three-dimensional model of the same (or closest) phase as the ultrasound image obtained by the kth scan, is the mth three-dimensional model with Matching grid points, Ω m is the set of grid points of the mth three-dimensional model.

[0073] The objective function (2) also means to find the minimum value of the sum of the first distances. By optimizing the objective function (2), the transformation parameter S that minimizes the sum of the first distances can be used as the registration parameter S. * .

[0074] The above objective function (1) and objective function (2) are equivalent in the solution process and can be transformed into a nonlinear least squares problem. Its nonlinear characteristics originate from the special transformation group, which is a subgroup of the general linear group, namely the Lie group. By performing linearization on the local Lie algebra, the gradient information can be obtained, and then the exponential mapping function is used to convert the gradient information back to the space of the special transformation group. The iterative solution method is used to gradually make the sum of the first distances approach the minimum value, and finally determine the registration parameter S * .

[0075] Get the registration parameter S * Afterwards, the three-dimensional model to be registered can be registered with the two-dimensional ultrasound image to be registered.

[0076] In the fixed mode, the deformed 3D model can be expressed as p i ′ =S * p i .

[0077] In dynamic mode, the deformed 3D model can be expressed as

[0078] In the implementation of the present application, a group of ultrasound image sequences can be registered, so that the obtained registration parameters can be applied to each ultrasound image in the ultrasound image sequence, without the need to register each ultrasound image one by one, and the registration efficiency is higher. During the registration process, the phase of each ultrasound image to be registered in the ultrasound image sequence is taken into account, and the most suitable three-dimensional model is matched for it, so that the registration parameters can be more accurate.

[0079] Please refer to Figure 3 , Figure 3 The red part in the figure represents the heart chamber obtained by segmenting the medical image. 3-1 to 3-4 are four groups of comparison images. In each group of comparison images, the first row shows the original result without using the registration scheme provided by this application, and the green part shows the cross-sectional view of the three-dimensional model to be registered on the plane of the ultrasound image. The second row shows the effect after being processed by the registration scheme provided by this application, and the green part shows the cross-sectional view of the deformed three-dimensional model on the plane of the ultrasound image. Figure 3 It can be clearly seen in each set of comparison images that the deformed 3D model fits more closely with the heart chamber represented by the red part in the cross-sectional view on the plane of the ultrasound image (the green part in the second row), especially showing a significant effect in scale optimization. Actual tests found that the average target registration error was reduced from 12.61 mm to 11.60 mm, indicating a significant improvement in registration accuracy.

[0080] In some embodiments of the present application, after determining the deformed three-dimensional model, the method may further include: displaying the deformed three-dimensional model to facilitate screening by doctors.

[0081] In some other embodiments of the present application, after determining the deformed three-dimensional model, the method may further include: performing surgical navigation according to the deformed three-dimensional model. The surgical navigation is used to display surgical instruments in the deformed three-dimensional model, so that the doctor can know the relative position between the current surgical instrument and the scanned object by viewing the deformed three-dimensional model.

[0082] It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders.

[0083] like Figure 4 The figure is a schematic structural diagram of an ultrasonic three-dimensional model registration device 400 provided in an embodiment of the present application. The ultrasonic three-dimensional model registration device 400 is configured on an electronic device.

[0084] Specifically, the ultrasound three-dimensional model registration device 400 may include:

[0085] An acquisition unit 401 is used to acquire an ultrasound image to be registered and a three-dimensional model to be registered;

[0086] A parameter determination unit 402 is used to determine the registration parameters between the ultrasound image to be registered and the three-dimensional model to be registered, wherein the registration parameters include a scale parameter, a rotation matrix and a translation vector of the three-dimensional model to be registered, and the scale parameter is used to scale the size of the three-dimensional model to be registered;

[0087] The registration unit 403 is used to register the three-dimensional model to be registered with the ultrasound image to be registered according to the registration parameters, and determine the deformed three-dimensional model.

[0088] In some embodiments of the present application, the parameter determination unit 402 is specifically used to: determine matching point pairs, each of the matching point pairs includes a contour point in the ultrasound image to be registered and a grid point on the three-dimensional model to be registered that matches the contour point; use a transformation parameter that minimizes the sum of each first distance as the registration parameter, each of the first distances being the distance between the grid point and the contour point transformed by the transformation parameter in one of the matching point pairs.

[0089] In some embodiments of the present application, the parameter determination unit 402 is specifically used to: associate the ultrasound images to be registered in different phases with a three-dimensional model to be registered respectively; match the contour points in the ultrasound images to be registered in each phase with the grid points on the associated three-dimensional model to be registered to obtain the matching point pairs.

[0090] In some embodiments of the present application, the parameter determination unit 402 is specifically used to: associating the ultrasound images to be registered at different phases with a three-dimensional model to be registered, respectively, including: if there are three-dimensional models to be registered at different phases, then according to the phase of each ultrasound image to be registered and the phase of each three-dimensional model to be registered, associating the ultrasound images to be registered and the three-dimensional models to be registered.

[0091] In some embodiments of the present application, the parameter determination unit 402 is specifically used to: associate the ultrasound images to be registered in different phases with a three-dimensional model to be registered respectively, including: if there is only a three-dimensional model to be registered in a single phase, associating all the ultrasound images to be registered with the same three-dimensional model to be registered.

[0092] In some embodiments of the present application, the parameter determination unit 402 is specifically used to: use the transformation parameter that will minimize the sum of each first distance as the alignment parameter, including: correcting each first distance using a robustness loss function; and use the transformation parameter that will minimize the sum of each corrected first distance as the alignment parameter.

[0093] In some embodiments of the present application, the apparatus 400 for ultrasound three-dimensional model registration may further include a display unit for displaying the deformed three-dimensional model.

[0094] In some embodiments of the present application, the ultrasound three-dimensional model registration apparatus 400 may further include a surgical navigation unit for performing surgical navigation according to the deformed three-dimensional model.

[0095] It should be noted that, for the convenience and simplicity of description, the specific working process of the above-mentioned ultrasound three-dimensional model registration device 400 can be referred to Figures 1 to 3 The corresponding process of the method will not be repeated here.

[0096] like Figure 5FIG. 5 is a schematic diagram of an electronic device 5 provided in an embodiment of the present application. Specifically, the electronic device 5 may include: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50, such as a registration program. When the processor 50 executes the computer program 52, the steps in the above-mentioned method embodiments for registration of the three-dimensional ultrasound model are implemented, such as Figure 1 Alternatively, when the processor 50 executes the computer program 52, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Figure 4 The functions of the acquisition unit 401, the parameter determination unit 402 and the registration unit 403 are shown.

[0097] The computer program may be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the electronic device 5.

[0098] For example, the computer program can be divided into: an acquisition unit, a parameter determination unit and a registration unit. The specific functions of each unit are as follows: an acquisition unit, used to acquire the ultrasound image to be registered and the three-dimensional model to be registered; a parameter determination unit, used to determine the registration parameters between the ultrasound image to be registered and the three-dimensional model to be registered, the registration parameters include the scale parameters, rotation matrix and translation vector of the three-dimensional model to be registered, and the scale parameters are used to scale the size of the three-dimensional model to be registered; a registration unit, used to register the three-dimensional model to be registered to the ultrasound image to be registered according to the registration parameters, and determine the deformed three-dimensional model.

[0099] The electronic device 5 may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will appreciate that Figure 5 It is only an example of the electronic device 5 and does not constitute a limitation of the electronic device 5. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 5 may also include input and output devices, network access devices, buses, etc.

[0100] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0101] The memory 51 may be an internal storage unit of the electronic device 5, such as a hard disk or memory of the electronic device 5. The memory 51 may also be an external storage device of the electronic device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 5. Further, the memory 51 may also include both an internal storage unit of the electronic device 5 and an external storage device. The memory 51 is used to store the computer program and other programs and data required by the electronic device 5. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0102] It should be noted that, for the convenience and brevity of description, the structure of the electronic device 5 can also refer to the specific description of the structure in the method embodiment, which will not be repeated here.

[0103] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0104] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0105] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0106] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0107] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0108] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0109] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0110] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for ultrasound three-dimensional model registration, characterized in that: include: Acquire the ultrasound image to be registered and the three-dimensional model to be registered; Determine the registration parameters between the ultrasound image to be registered and the three-dimensional model to be registered, wherein the registration parameters include a scale parameter, a rotation matrix and a translation vector of the three-dimensional model to be registered, and the scale parameter is used to scale the size of the three-dimensional model to be registered; According to the registration parameters, the three-dimensional model to be registered is registered with the ultrasound image to be registered to determine a deformed three-dimensional model.

2. The method for ultrasound three-dimensional model registration according to claim 1, characterized in that: The determining of the registration parameters between the ultrasound image to be registered and the three-dimensional model to be registered comprises: Determine matching point pairs, each matching point pair comprising a contour point in the ultrasound image to be registered and a grid point on the three-dimensional model to be registered that matches the contour point; The transformation parameter that minimizes the sum of the first distances is used as the registration parameter, and each of the first distances is the distance between the grid point and the contour point transformed by the transformation parameter in a matching point pair.

3. The method for ultrasound three-dimensional model registration as claimed in claim 2, characterized in that: The determining of the matching point pairs comprises: Associating the to-be-registered ultrasound images of different phases with a to-be-registered three-dimensional model respectively; The contour points in the ultrasound image to be registered in each phase are matched with the grid points on the associated three-dimensional model to be registered to obtain the matching point pairs.

4. The method for ultrasound three-dimensional model registration as claimed in claim 3, characterized in that: The step of associating the ultrasound images to be registered at different phases with a three-dimensional model to be registered, respectively, comprises: If there are three-dimensional models to be registered in different phases, the ultrasound images to be registered and the three-dimensional models to be registered are associated according to the phase of each ultrasound image to be registered and the phase of each three-dimensional model to be registered.

5. The method for ultrasound three-dimensional model registration as claimed in claim 3, characterized in that: The step of associating the ultrasound images to be registered at different phases with a three-dimensional model to be registered, respectively, comprises: If there is only a single phase of the three-dimensional model to be registered, all the ultrasound images to be registered are associated with the same three-dimensional model to be registered.

6. The method for ultrasound three-dimensional model registration as claimed in claim 2, characterized in that: The transformation parameter that minimizes the sum of the first distances as the registration parameter includes: Using a robustness loss function to correct each of the first distances; The transformation parameter that minimizes the sum of the corrected first distances is used as the registration parameter.

7. The method for ultrasound three-dimensional model registration according to any one of claims 1 to 6, characterized in that: After determining the deformed three-dimensional model, the method further includes: displaying the deformed three-dimensional model; And / or, performing surgical navigation based on the deformed three-dimensional model.

8. A device for ultrasound three-dimensional model registration, characterized in that: include: An acquisition unit, used for acquiring an ultrasound image to be registered and a three-dimensional model to be registered; a parameter determination unit, configured to determine registration parameters between the ultrasound image to be registered and the three-dimensional model to be registered, wherein the registration parameters include a scale parameter, a rotation matrix and a translation vector of the three-dimensional model to be registered, and the scale parameter is used to scale the size of the three-dimensional model to be registered; The registration unit is used to register the three-dimensional model to be registered with the ultrasound image to be registered according to the registration parameters, and determine the deformed three-dimensional model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for ultrasound three-dimensional model registration according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The invention comprises a computer program, which, when being executed, enables the method for ultrasound three-dimensional model registration as claimed in any one of claims 1 to 7 to be executed.

Images