3D positioning and annotation method, device, equipment and medium for intracardiac ultrasound images

Abstract

The embodiment of the present invention provides a 3D positioning and labeling method, device, equipment and medium for intracardiac ultrasound images, wherein the method includes: determining a heart model that matches the heart-related information of a target object; constructing a three-dimensional space for the determined heart model; processing each two-dimensional intracardiac ultrasound image of the target object to obtain a contour image of the heart tissue; determining a preset sector in the three-dimensional space, converting the three-dimensional data of the preset sector into two-dimensional data and matching it with the contour image to obtain a matching area; calibrating the position of the calibration target in the two-dimensional intracardiac ultrasound image in the three-dimensional space according to the relevant position of the sector in the matching area in the three-dimensional space. The scheme can conveniently and efficiently realize the 3D positioning and labeling of intracardiac ultrasound images, which is conducive to improving the accuracy, applicability and promotion of positioning and labeling.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a 3D positioning and labeling method, device, equipment and medium for intracardiac ultrasound images. Background Art

[0002] In modern medicine, intracardiac echocardiography (ICE) plays an important role in cardiology. ICE images are now an important means of providing reference guidance for atrial fibrillation ablation, transcatheter patent foramen ovale closure, left atrial appendage closure and other surgeries. ICE images can reflect the cardiac tissue structure and myocardial movement in the heart cavity in real time, but because ICE is introduced into the heart through a catheter through the blood vessels, it can only reflect the situation of a certain sector structure of the heart. This feature of ICE images restricts cardiologists from needing a lot of experience and knowledge to supplement the real-time heart information given by ultrasound images during surgery, especially the movement position of the ICE catheter and image in the actual 3D structure of the heart, which feels unknown and uncertain because it is in the body.

[0003] In current intracardiac ultrasound instruments or software, many products use magnetic field positioning to assist in confirming ICE catheters. The conditions for use are strict, and magnetic sheets need to be customized for the catheters, and there must be no other magnetic signal interference in the environment. With the rapid development of artificial intelligence technology today, the use of convolutional neural network methods to recognize ICE images is a very effective method for locating intraoperative ICE catheters and determining the image sector position and cardiac tissue structure. However, the convolutional neural network method is a supervised algorithm that requires a large amount of labeled data for training. There is currently no corresponding effective labeling and positioning method for using 2D ultrasound images to locate the spatial situation in the 3D cardiac structure. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a 3D positioning and labeling method for intracardiac ultrasound images to solve the technical problem in the prior art that the spatial situation of 2D ultrasound images in 3D cardiac structures cannot be labeled. The method includes:

[0005] Determining a heart model that matches the heart-related information of the target subject;

[0006] Constructing a three-dimensional space for the determined heart model;

[0007] Processing each intracardiac ultrasonic two-dimensional image of the target object to obtain a contour image of the heart tissue;

[0008] Determine a preset sector in the three-dimensional space, convert the three-dimensional data of the preset sector into two-dimensional data, and then match it with the contour image to obtain a matching area, wherein the preset sector includes the cardiac tissue structure in the contour image;

[0009] The position of the calibration target in the intracardiac ultrasound two-dimensional image in the three-dimensional space is calibrated according to the relative position of the sector of the matching area in the three-dimensional space.

[0010] The embodiment of the present invention also provides a 3D positioning and labeling device for intracardiac ultrasound images to solve the technical problem that the spatial situation of 2D ultrasound images in 3D cardiac structures cannot be labeled in the prior art. The device includes:

[0011] A model determination module, used to determine a heart model that matches the heart-related information of the target object;

[0012] A space construction module, used for constructing a three-dimensional space for a determined heart model;

[0013] An image processing module, used for processing each intracardiac ultrasonic two-dimensional image of the target object to obtain a contour image of the heart tissue;

[0014] a matching module, configured to determine a preset sector in the three-dimensional space, convert the three-dimensional data of the preset sector into two-dimensional data, and then match the data with the contour image to obtain a matching area, wherein the preset sector includes the cardiac tissue structure in the contour image;

[0015] The calibration module is used to calibrate the position of the calibration target in the intracardiac ultrasound two-dimensional image in the three-dimensional space according to the relative position of the sector of the matching area in the three-dimensional space.

[0016] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned 3D positioning and labeling methods for intracardiac ultrasound images when executing the computer program, so as to solve the technical problem in the prior art that the spatial situation of a 2D ultrasound image in a 3D cardiac structure cannot be labeled.

[0017] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program for executing any of the above-mentioned 3D positioning and labeling methods of intracardiac ultrasound images, so as to solve the technical problem in the prior art that the spatial situation of 2D ultrasound images in 3D cardiac structures cannot be labeled.

[0018] Compared with the prior art, the beneficial effects achieved by at least one of the above technical solutions adopted in the embodiments of this specification include at least: it is proposed to construct a three-dimensional space based on a heart model adapted to the heart-related information of the target object, and to process each two-dimensional intracardiac ultrasound image of the target object to obtain a contour image of the heart tissue, and then determine a preset sector including the heart tissue structure in the contour image in the three-dimensional space, and finally, convert the three-dimensional data of the preset sector into two-dimensional data and match it with the contour image to obtain a matching area, and the position of the calibration target in the two-dimensional intracardiac ultrasound image in the three-dimensional space can be calibrated according to the relevant position of the matching area in the three-dimensional space, that is, indirectly or equivalently calibrating the position of the calibration target in the two-dimensional intracardiac ultrasound image in the real 3D heart structure. This positioning and labeling method can conveniently and efficiently realize the 3D positioning and labeling of intracardiac ultrasound images, does not require structures such as magnetic sheets, is not affected by environmental factors, and is conducive to improving the accuracy, applicability and promotion of positioning and labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of a 3D positioning and annotation method of an intracardiac ultrasound image provided by an embodiment of the present invention;

[0021] Figure 2 is a schematic diagram of the principle of a 3D positioning and annotation method for intracardiac ultrasound images provided by an embodiment of the present invention;

[0022] Figure 3 is a schematic diagram of a 3D Octree-like space of a heart model provided by an embodiment of the present invention;

[0023] Figure 4 It is a schematic diagram of a fan range in a 3D Octree-like space provided by an embodiment of the present invention;

[0024] Figure 5 is a schematic diagram of matching an intracardiac ultrasound two-dimensional image with a template contour provided by an embodiment of the present invention;

[0025] Figure 6 is a flow chart of a method for implementing the above-mentioned 3D positioning and annotation method of intracardiac ultrasound images provided by an embodiment of the present invention;

[0026] Figure 7is a structural block diagram of a computer device provided by an embodiment of the present invention;

[0027] Figure 8 It is a structural block diagram of a 3D positioning and annotation device for intracardiac ultrasound images provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0029] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0030] In an embodiment of the present invention, a 3D positioning and annotation method for intracardiac ultrasound images is provided. Figure 1 As shown, the method includes:

[0031] Step S101: determining a heart model that matches the heart-related information of the target object;

[0032] Step S102: constructing a three-dimensional space for the determined heart model;

[0033] Step S103: Process each intracardiac ultrasonic two-dimensional image of the target object to obtain a contour image of the heart tissue;

[0034] Step S104: determining a preset sector in the three-dimensional space, converting the three-dimensional data of the preset sector into two-dimensional data and matching the data with the contour image to obtain a matching area, wherein the preset sector includes the cardiac tissue structure in the contour image;

[0035] Step S105: calibrating the position of the calibration target in the intracardiac ultrasound two-dimensional image in the three-dimensional space according to the relative position of the sector of the matching area in the three-dimensional space.

[0036] Depend on Figure 1As can be seen from the process shown, in the embodiment of the present invention, it is proposed to construct a three-dimensional space based on a heart model that is compatible with the heart-related information of the target object, and process each intracardiac ultrasound two-dimensional image of the target object to obtain a contour image of the heart tissue, and then determine a preset sector including the heart tissue structure in the contour image in the three-dimensional space, and finally, convert the three-dimensional data of the preset sector into two-dimensional data and match it with the contour image to obtain a matching area, and the position of the calibration target in the intracardiac ultrasound two-dimensional image in the three-dimensional space can be calibrated according to the relevant position of the matching area in the three-dimensional space, that is, indirectly or equivalently calibrating the position of the calibration target in the intracardiac ultrasound two-dimensional image in the real 3D heart structure. This positioning and labeling method can conveniently and efficiently realize the 3D positioning and labeling of intracardiac ultrasound images, does not require structures such as magnetic sheets, is not affected by environmental factors, and is conducive to improving the accuracy, applicability, and promotion of positioning and labeling.

[0037] In a specific implementation, the target object may be any owner of a heart for which 3D positioning annotation of an intracardiac ultrasound image is required, for example, a patient or other person undergoing intracardiac ultrasound.

[0038] In a specific implementation, the above-mentioned heart-related information may be any information related to the heart tissue structure, such as the size and shape of the heart. Specifically, the process of determining a heart model that matches the heart-related information may be a process of building a 3D heart model based on the heart-related information, or a process of matching and selecting an existing 3D heart model in a database through the heart-related information, such as Figure 2 shown.

[0039] In specific implementation, the process of processing each intracardiac ultrasound two-dimensional image to obtain the contour image of the heart tissue can be implemented by any method that can obtain a 2D contour image, and the entire image needs to be fully labeled, that is, the myocardial tissue and chambers on all ultrasound sectors are distinguished to display the information of the two-dimensional ultrasound heart tissue contour. For example, the traditional threshold segmentation algorithm can be used to process the image.

[0040] In the specific implementation, after obtaining the 2D contour image of the intracardiac ultrasound 2D image, in order to ensure the accuracy of the positioning calibration result, it is proposed that before matching the 2D contour image with the data in the 3D space, Figure 2 As shown, the 2D contour image is corrected to ensure the accuracy of the 2D contour image. For example, after converting the three-dimensional data of the preset sector into two-dimensional data and before matching it with the contour image, the contour lines in the contour image are converted into polygons, and the vertices in the polygons are corrected to correct the contour image.

[0041] Specifically, it is possible to determine whether the 2D contour image is correct manually or by related software algorithms. If not, the 2D contour image can be corrected. During the correction process, the vertices in the polygon can be deleted, modified, and other correction operations can be performed to achieve the purpose of correcting and modifying the contour image.

[0042] In the specific implementation, after the heart model is determined, in order to accurately and conveniently obtain the position of the contour image in the 3D heart structure, a 3D class Octree space for constructing the heart model is proposed, such as Figure 3 As shown, the relevant sectors of the 3D Octree-like space of the heart model are then matched with the contour image to locate and calibrate the position of the contour image in the 3D Octree-like space of the heart model. Specifically, in the process of constructing the 3D Octree-like space of the heart model, the 3D Octree-like space can be directly constructed based on the heart model without distinguishing the depth. However, in order to improve the accuracy of matching, positioning and calibration, it is proposed to construct 3D Octree-like spaces of different depths for a determined heart model, wherein different levels of tissue structures of the heart model in the 3D Octree-like space have different depth values.

[0043] Specifically, the principle process of constructing 3D Octree-like spaces of different depths is to supplement the positions of other non-model voxels on the basis of the 3D structure constructed by the heart model voxels, and tissue structures of different depths correspond to different depth values, so that the positions of catheters and fans can be matched and located more finely and accurately based on different depth values.

[0044] Specifically, in the process of constructing 3D Octree-like spaces of different depths, the size and number of depth values ​​can be determined based on factors such as the tissue structure characteristics and size of the heart model and the accuracy requirements of positioning calibration. For example, the deeper the position of the tissue structure in the heart model, the larger the corresponding depth value; the more depth values ​​there are, the finer the 3D space of the tissue structure is divided and constructed, and thus the matching and positioning calibration positions are also finer and more accurate.

[0045] In the specific implementation, in order to improve the efficiency and accuracy of the matching process, it is proposed to first determine the preset sector in the 3D Octree space, such as Figure 4 As shown, the preset sector includes the cardiac tissue structure in the contour image, that is, the approximate target matching sector is first determined, and then the data of the preset sector is precisely matched with the contour image in a targeted manner to reduce the amount of matching calculations, increase the matching speed, and improve the matching accuracy.

[0046] In order to improve the accuracy of positioning and marking in the specific implementation, in this embodiment, it is proposed to convert the three-dimensional data of the preset sector into two-dimensional data and then match it with the contour image, including:

[0047] After the three-dimensional data of the preset sector is converted into two-dimensional data, the two-dimensional data is matched with the contour image in the progressive order of the depth information. That is, the two-dimensional data with the same depth value are matched with the contour image in sequence in the progressive order of the depth information. For example, the depth values ​​of the 3D Octree space of the heart model include 1, 2, 3, etc. The deeper the position of the tissue structure, the greater the depth value corresponding to it. Therefore, the two-dimensional data with a depth value of 1 is matched with the contour image first, and then the two-dimensional data with a depth value of 2 is matched with the contour image, and then the two-dimensional data with a depth value of 3 is matched with the contour image, and so on.

[0048] In specific implementation, in order to further improve the accuracy of positioning and calibration, it is proposed to determine multiple preset sectors, take each preset sector as a template contour, convert the three-dimensional data of each template contour into two-dimensional data, match the two-dimensional data of each template contour with the contour image in the progressive order of depth information, and determine the template contour with the highest matching degree as the matching area. Figure 5 As shown, based on the relevant cardiac tissue structure in the intracardiac ultrasound two-dimensional image of Figure (a), the fan region including the corresponding cardiac tissue structure can be pre- and roughly determined in the 3D Octree-like space of the heart model, such as Figure 5 The dark sector area shown in Figure (b) has a central angle corresponding to the position of the catheter. Such a sector area can be used as a template contour to be finely matched with the contour image, and the template contour with the highest matching degree (such as similarity, etc.) is determined among multiple template contours.

[0049] Specifically, in the process of positioning and calibrating by matching the contour image with the data of the 3D Octree-like space of the heart model, it is possible to realize positioning and calibration of any calibration target in the two-dimensional intracardiac ultrasound image. For example, the calibration target can be the current sector of the two-dimensional intracardiac ultrasound image and the catheter of the intracardiac ultrasound, or it can be one or more specified cardiac tissue structures in the two-dimensional intracardiac ultrasound image. For example, when the calibration target is the current sector of the two-dimensional intracardiac ultrasound image and the catheter of the intracardiac ultrasound, the vertex position of the sector of the matching area in the three-dimensional space (such as the 3D Octree-like space) is calibrated as the position of the sector corresponding to the two-dimensional intracardiac ultrasound image in the three-dimensional space, and the center position of the sector of the matching area in the three-dimensional space is calibrated as the position of the two-dimensional ultrasound catheter in the three-dimensional space.

[0050] In specific implementation, after obtaining the positioning calibration result, the positioning calibration result can also be corrected, for example, Figure 2As shown, if the positions of the three points in the positioning and calibration results (the center of the sector and the points on the arc of the two radius sides) are judged to be inaccurate, the position of the moving catheter can be re-matched and repositioned to make slight adjustments to these three points to reach the ideal position.

[0051] In specific implementation, the above-mentioned 3D positioning and labeling method of intracardiac ultrasound images can be applied to any application scenario that needs to determine the relevant 3D position of an intracardiac ultrasound two-dimensional image. For example, the above-mentioned 3D positioning and labeling method of intracardiac ultrasound images can be used to feedback the relevant 3D position of intracardiac ultrasound in real time or statically, so as to provide accurate data basis for scenarios such as catheter operation of intracardiac ultrasound, cardiology surgery operation, etc.; it can also provide sample data for the training of neural network models, for example, obtain the position of the calibration target in the three-dimensional space in multiple intracardiac ultrasound two-dimensional images, and use the position of each calibration target in the three-dimensional space as the label corresponding to the intracardiac ultrasound two-dimensional image; use the multiple intracardiac ultrasound two-dimensional images and the corresponding labels as samples, train the neural network model through the samples, and determine the position of the calibration target in the three-dimensional space in a new intracardiac ultrasound two-dimensional image through the trained neural network model.

[0052] In specific implementation, the following is a flowchart of the specific implementation of the above-mentioned 3D positioning and annotation method of intracardiac ultrasound images. Figure 6 As shown, the process may include the following steps:

[0053] 1. Obtaining a two-dimensional ultrasonic image (step 1001): The two-dimensional ultrasonic image may be obtained from a two-dimensional ultrasonic image database at the back end, or may be obtained from a two-dimensional ultrasonic image acquisition device or generation device.

[0054] 2. Start to obtain the contour of the two-dimensional image through the traditional threshold segmentation algorithm (step 10021). It uses an algorithm such as threshold in opencv to perform binary threshold segmentation, and then judge whether the segmentation is accurate or not (step 10022). If the segmentation is inaccurate, enter the manual correction to correct the image contour and the segmentation contour of the heart tissue (step 10023). After the algorithm segmentation is accurate or the manual correction is completed, enter the next step.

[0055] 3. At the same time, it is necessary to obtain the corresponding 3D heart model based on other information of the two-dimensional ultrasound image, such as case information (step 10031), that is, to construct an Octree-like space corresponding to the 3D heart model (step 10032), and select the depth of the Octree space according to business needs (the greater the depth, the finer the space division will be).

[0056] 4. Then, by searching for three points in a sector in a 3D simulation space that approximates an ultrasound sector, namely, the center of the sector and the points on the arc of the two radii of the sector, as node coordinate points for determining the position (step 1004). The heart section cut by the sector determined by the three points will also have a similar structure diagram of the segmentation contour style. This method uses the stencil buffer contour diagram of the cut section in the 3D space as the segmentation contour style, such as Figure 5 After the stencilbuffer outline is mapped from 3D data to a two-dimensional plane, a matching search is performed with the results of step 10022 and step 10023, that is, a position neighbor search matching is performed (step 1005).

[0057] 5. If the pairing is correct (step 1006), the 3-point position required by the business is found, otherwise manual error correction is performed to manually modify the 3-point position of the sector-type Octree position (step 1007).

[0058] 6. Finally, the determined three-point positions serve as the annotation node positions of the two-dimensional ultrasound image data required by the artificial intelligence algorithm, and can be used to learn and determine the approximate positions of the two-dimensional ultrasound catheter and its fan where the unknown two-dimensional ultrasound image is located in the future.

[0059] In this embodiment, a computer device is provided, such as Figure 7 As shown, it includes a memory 701, a processor 702, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the above-mentioned 3D positioning and labeling methods for intracardiac ultrasound images is implemented.

[0060] Specifically, the computer device may be a computer terminal, a server or a similar computing device.

[0061] In this embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program for executing any of the above-mentioned 3D positioning and labeling methods for intracardiac ultrasound images.

[0062] Specifically, computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0063] Based on the same inventive concept, an embodiment of the present invention also provides a 3D positioning and annotation device for intracardiac ultrasound images, as described in the following embodiments. Since the principle of solving the problem by the 3D positioning and annotation device for intracardiac ultrasound images is similar to that of the 3D positioning and annotation method for intracardiac ultrasound images, the implementation of the 3D positioning and annotation device for intracardiac ultrasound images can refer to the implementation of the 3D positioning and annotation method for intracardiac ultrasound images, and the repeated parts will not be repeated. As used below, the terms "unit" or "module" can be a combination of software and / or hardware that implements predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0064] Figure 8 is a structural block diagram of a 3D positioning and annotation device for intracardiac ultrasound images according to an embodiment of the present invention. Figure 8 As shown, the device comprises:

[0065] A model determination module 801 is used to determine a heart model that matches the heart-related information of the target object;

[0066] A space construction module 802 is used to construct a three-dimensional space for the determined heart model;

[0067] An image processing module 803 is used to process each intracardiac ultrasonic two-dimensional image of the target object to obtain a contour image of the heart tissue;

[0068] A matching module 804 is used to determine a preset sector in the three-dimensional space, convert the three-dimensional data of the preset sector into two-dimensional data, and then match it with the contour image to obtain a matching area, wherein the preset sector includes the cardiac tissue structure in the contour image;

[0069] The calibration module 805 is used to calibrate the position of the calibration target in the intracardiac ultrasound two-dimensional image in the 3D Octree-like space according to the relative position of the sector of the matching area in the three-dimensional space.

[0070] In one embodiment, the space construction module is used to construct 3D Octree-like spaces of different depths for the determined heart model, wherein different levels of organizational structures of the heart model in the 3D Octree-like space have different depth values.

[0071] In one embodiment, the matching module is used to convert the three-dimensional data of the preset sector into two-dimensional data, and then match the two-dimensional data with the contour image in a progressive order of depth information.

[0072] In one embodiment, a matching module is used to determine a plurality of preset sectors, take each of the preset sectors as a template contour, convert the three-dimensional data of each template contour into two-dimensional data, and then match the two-dimensional data of each template contour with the contour image in a progressive order of depth information, and determine the template contour with the highest matching degree as the matching area.

[0073] In one embodiment, the above device further comprises:

[0074] The correction module is used to convert the contour lines in the contour image into polygons and correct the vertices in the polygons to correct the contour image before matching the contour image after converting the three-dimensional data of the preset sector into two-dimensional data.

[0075] In one embodiment, a calibration module is used to calibrate the vertex position of the fan surface of the matching area in the three-dimensional space as the position of the fan surface corresponding to the two-dimensional intracardiac ultrasound image in the three-dimensional space, and to calibrate the center position of the fan surface of the matching area in the three-dimensional space as the position of the two-dimensional ultrasound catheter in the three-dimensional space when the calibration target is the two-dimensional ultrasound catheter and the corresponding fan surface where the two-dimensional intracardiac ultrasound image is located.

[0076] In one embodiment, the above device further comprises:

[0077] The model training module is used to obtain the position of the calibration target in the multiple two-dimensional intracardiac ultrasound images in the three-dimensional space, and use the position of each calibration target in the three-dimensional space as a label corresponding to the two-dimensional intracardiac ultrasound image; use the multiple two-dimensional intracardiac ultrasound images and the corresponding labels as samples, train a neural network model through the samples, and determine the position of the calibration target in the three-dimensional space in a new two-dimensional intracardiac ultrasound image through the trained neural network model.

[0078] The embodiment of the present invention achieves the following technical effects: it proposes to construct a three-dimensional space based on a heart model that is compatible with the heart-related information of the target object, and to process each two-dimensional intracardiac ultrasound image of the target object to obtain a contour image of the heart tissue, and then determine a preset sector including the heart tissue structure in the contour image in the three-dimensional space, and finally, convert the three-dimensional data of the preset sector into two-dimensional data and match it with the contour image to obtain a matching area, and the position of the calibration target in the two-dimensional intracardiac ultrasound image in the 3D Octree space can be calibrated according to the relevant position of the matching area in the three-dimensional space, that is, indirectly or equivalently calibrating the position of the calibration target in the two-dimensional intracardiac ultrasound image in the real 3D heart structure. The positioning and labeling method can conveniently and efficiently realize the 3D positioning and labeling of intracardiac ultrasound images, does not require structures such as magnetic sheets, is not affected by environmental factors, and is conducive to improving the accuracy, applicability, and promotion of positioning and labeling.

[0079] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A 3D positioning and annotation method for intracardiac ultrasound images, characterized in that: include: Determining a heart model that matches the heart-related information of the target subject; Constructing a three-dimensional space for the determined heart model; Processing each intracardiac ultrasonic two-dimensional image of the target object to obtain a contour image of the heart tissue; Determine a preset sector in the three-dimensional space, convert the three-dimensional data of the preset sector into two-dimensional data, and then match it with the contour image to obtain a matching area, wherein the preset sector includes the cardiac tissue structure in the contour image; Calibrate the position of the calibration target in the intracardiac ultrasound two-dimensional image in the three-dimensional space according to the relative position of the sector of the matching area in the three-dimensional space; The method further comprises: When the calibration target is the two-dimensional ultrasound catheter and the corresponding fan surface where the two-dimensional intracardiac ultrasound image is located, the vertex position of the fan surface of the matching area in the three-dimensional space is calibrated as the position of the fan surface corresponding to the two-dimensional intracardiac ultrasound image in the three-dimensional space, and the center position of the fan surface of the matching area in the three-dimensional space is calibrated as the position of the two-dimensional ultrasound catheter in the three-dimensional space.

2. The 3D positioning and annotation method of intracardiac ultrasound images according to claim 1, characterized in that: Construct a three-dimensional space for the determined heart model, including: A 3D Octree-like space of different depths is constructed for the determined heart model, wherein different levels of tissue structures of the heart model in the 3D Octree-like space have different depth values.

3. The 3D positioning and labeling method of intracardiac ultrasound images according to claim 2, characterized in that: Converting the three-dimensional data of the preset sector into two-dimensional data and matching it with the contour image comprises: After the three-dimensional data of the preset sector is converted into two-dimensional data, the two-dimensional data is matched with the contour image in a progressive order of depth information.

4. The 3D positioning and labeling method of intracardiac ultrasound images according to claim 3, characterized in that: After converting the three-dimensional data of the preset sector into two-dimensional data, matching the two-dimensional data with the contour image in a progressive order of depth information includes: Determine a plurality of preset sectors, take each of the preset sectors as a template contour, convert the three-dimensional data of each template contour into two-dimensional data, match the two-dimensional data of each template contour with the contour image in a progressive order of depth information, and determine the template contour with the highest matching degree as the matching area.

5. The 3D positioning and labeling method of intracardiac ultrasound images according to any one of claims 1 to 4, characterized in that: Also includes: After converting the three-dimensional data of the preset sector into two-dimensional data and before matching with the contour image, the contour lines in the contour image are converted into polygons, and the vertices in the polygons are corrected to correct the contour image.

6. The 3D positioning and labeling method of intracardiac ultrasound images according to any one of claims 1 to 4, characterized in that: Also includes: Acquire the positions of the calibration targets in the three-dimensional space in the plurality of the two-dimensional intracardiac ultrasound images, and use the position of each calibration target in the three-dimensional space as a label corresponding to the two-dimensional intracardiac ultrasound image; A plurality of the intracardiac ultrasound two-dimensional images and the corresponding labels are used as samples, a neural network model is trained by the samples, and the position of the calibration target in the new intracardiac ultrasound two-dimensional image in the three-dimensional space is determined by the trained neural network model.

7. A 3D positioning and annotation device for intracardiac ultrasound images, characterized in that: include: A model determination module, used to determine a heart model that matches the heart-related information of the target object; A space construction module, used for constructing a three-dimensional space for a determined heart model; An image processing module, used for processing each intracardiac ultrasonic two-dimensional image of the target object to obtain a contour image of the heart tissue; a matching module, configured to determine a preset sector in the three-dimensional space, convert the three-dimensional data of the preset sector into two-dimensional data, and then match the data with the contour image to obtain a matching area, wherein the preset sector includes the cardiac tissue structure in the contour image; A calibration module, used for calibrating the position of the calibration target in the intracardiac ultrasound two-dimensional image in the three-dimensional space according to the relative position of the sector of the matching area in the three-dimensional space; The calibration module is used to calibrate the vertex position of the fan surface of the matching area in the three-dimensional space as the position of the fan surface corresponding to the two-dimensional intracardiac ultrasound image in the three-dimensional space, and to calibrate the center position of the fan surface of the matching area in the three-dimensional space as the position of the two-dimensional ultrasound catheter in the three-dimensional space when the calibration target is the two-dimensional ultrasound catheter and the corresponding fan surface where the two-dimensional intracardiac ultrasound image is located.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the 3D positioning and labeling method of the intracardiac ultrasound image according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the 3D positioning and labeling method for intracardiac ultrasound images according to any one of claims 1 to 6.

Images