Three-dimensional surface reconstruction method and apparatus for two-dimensional intracardiac ultrasound catheter image

Abstract

The present invention belongs to the technical field of intracardiac ultrasound image processing technology. Provided are a three-dimensional surface reconstruction method and apparatus for a two-dimensional intracardiac ultrasound catheter image. The reconstruction method comprises the following steps: S1, collecting a plurality of consecutive ultrasound images inside a heart; S2, performing interpolation processing on the ultrasound images, so as to obtain a plurality of interpolated images; S3, inputting each ultrasound image into a pre-trained Unet model, so as to obtain a heart chamber segmentation image of each ultrasound image; S4, extracting edge information of a surface of each chamber of each heart chamber segmentation image, so as to obtain an edge image; S5, generating corresponding point cloud data on the basis of each edge image and interpolated image; and S6, performing surface reconstruction on the generated point cloud data by means of an Ashape algorithm, so as to obtain a three-dimensional model. The three-dimensional surface reconstruction method is an ultrasound image modeling method based on a two-dimensional image and a single ultrasound probe, supports heart chamber reconstruction updated in real time, realizes automatic position segmentation and modeling of a heart chamber, and has a higher speed and higher accuracy.

Description

A three-dimensional surface reconstruction method and device for two-dimensional intracardiac ultrasound catheter images Technical Field

[0001] The present invention relates to the technical field of echocardiogram image processing, and in particular to a method and device for three-dimensional surface reconstruction of a two-dimensional intracardiac ultrasound catheter image. Background Art

[0002] Intracardiac ultrasound catheter images are widely used in the diagnosis and treatment of cardiac diseases. However, reconstruction of these images remains a challenging task due to image noise and resolution limitations.

[0003] In complex scenarios, model reconstruction often relies on professional annotation, and its accuracy depends on the experience and ability of the annotators. This method is time-consuming and labor-intensive, resulting in the patient's optimal treatment window being missed. Therefore, automatic and accurate modeling is extremely important.

[0004] In recent years, numerous ultrasound image reconstruction methods have been proposed, but most have limitations. For example, intracardiac 3D ultrasound imaging catheters and systems, as well as methods for constructing 3D cardiac models, suffer from coarse models and the inability to reconstruct cardiac chambers. This makes it difficult to provide more accurate and comprehensive information for the diagnosis and treatment of cardiovascular diseases. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method, device, and electronic device for three-dimensional surface reconstruction of two-dimensional intracardiac ultrasound catheter images, to solve or partially solve the above-mentioned problems.

[0006] In a first aspect, an embodiment of the present invention provides a method for three-dimensional surface reconstruction of a two-dimensional intracardiac ultrasound catheter image, comprising the following steps:

[0007] S1: Multiple continuous ultrasound images are collected inside the heart through an intracardiac ultrasound catheter;

[0008] S2: performing interpolation processing on the blank areas between the ultrasound images to obtain a plurality of interpolated images;

[0009] S3: inputting each of the ultrasound images into a pre-trained Unet model to obtain an output cardiac chamber segmentation image of each of the ultrasound images;

[0010] S4: extracting edge information of the surface of each chamber of each cardiac chamber segmentation image to obtain an edge image of each chamber;

[0011] S5: generating corresponding point cloud data according to each of the edge images and the interpolation image;

[0012] S6: The generated point cloud data is reconstructed using the Ashape algorithm to obtain a three-dimensional model.

[0013] According to a specific implementation of the embodiment of the present invention, in step S1, the ultrasound image includes rotation angle information corresponding to the intracardiac ultrasound catheter.

[0014] According to a specific implementation of the embodiment of the present invention, in step S2, the interpolation process is linear interpolation, and the interpolated image includes pixel information and angle information.

[0015] According to a specific implementation of an embodiment of the present invention, in step S3, the Unet model training method is as follows:

[0016] S3.1: Prepare a dataset of intracardiac ultrasound images with corresponding cardiac chamber annotations;

[0017] S3.2: Preprocessing the intracardiac ultrasound images in the intracardiac ultrasound image dataset to remove noise, enhance contrast, and unify the scale of the images;

[0018] S3.3: Divide the preprocessed intracardiac ultrasound image dataset into a training set, a validation set, and a test set;

[0019] S3.4: Training the Unet model using the intracardiac ultrasound image data and the corresponding cardiac chamber annotations in the training set;

[0020] S3.5: Validate the Unet model during training using the validation set to evaluate model performance and perform fine-tuning.

[0021] S3.6: Perform a final evaluation on the trained and validated Unet model using the test set.

[0022] According to a specific implementation method of an embodiment of the present invention, in step S3.4, a cross entropy loss function is used to measure the difference between the prediction result and the true annotation, and the parameters of the model are optimized by back propagation. The training process uses a gradient descent algorithm to minimize the loss function.

[0023] According to a specific implementation of an embodiment of the present invention, step S3 further includes: the Unet model determines each of the output cardiac chamber separation images; if the determination is pending, manual intervention is performed to perform secondary annotation on the corresponding cardiac chamber separation image.

[0024] According to a specific implementation of the embodiment of the present invention, in step S4, the edge information is extracted based on the Sobel operator in the following manner:

[0025] For each pixel position (x, y) in the cardiac chamber segmentation image, the magnitude G and direction θ of the gradient can be calculated by the following formula:

[0026] ;

[0027] ;

[0028] Among them, G x and G y Represent the gradient of the pixels in the image in the x direction and y direction respectively.

[0029] According to a specific implementation of the embodiment of the present invention, in step S5, the point cloud data is generated as follows:

[0030] ;

[0031] in, is the coordinate of the point in the point cloud, (x, y) is the position of each pixel in the edge image or the interpolated image, θ is the angle information corresponding to the edge image or the interpolated image, and c is the offset constant.

[0032] This method embodiment has at least the following technical effects:

[0033] First, conventional cardiac cavity modeling currently relies on a second catheter, while this three-dimensional surface reconstruction method is an ultrasound image modeling method based on two-dimensional images and a single ultrasound probe, and based on the conversion of two-dimensional images into point clouds. This method reduces the operational process and difficulty, while supporting real-time updated cardiac chamber reconstruction, and can achieve automatic segmentation of cardiac cavity positions with faster speed and higher accuracy.

[0034] Second, in this 3D surface reconstruction method, the results of automatic segmentation are adjusted through human-computer interaction, and intervention is made at any time to ensure accuracy and reliability.

[0035] Third, in this 3D surface reconstruction method, after the cardiac chamber separation image is segmented, the boundary contour is extracted by edge and then converted into point cloud data, which greatly reduces the amount of calculation and improves efficiency.

[0036] In a second aspect, an embodiment of the present invention provides a three-dimensional surface reconstruction device for a two-dimensional intracardiac ultrasound catheter image, comprising:

[0037] a data acquisition module, configured to acquire a plurality of continuous ultrasound images inside the heart via an intracardiac ultrasound catheter;

[0038] An image interpolation module, configured to perform interpolation processing on blank areas between the ultrasound images to obtain a plurality of interpolated images;

[0039] A region segmentation module, configured to input each of the ultrasound images into a pre-trained Unet model to obtain an output cardiac chamber segmentation image of each of the ultrasound images;

[0040] an edge extraction module, configured to extract edge information of the surface of each chamber of each cardiac chamber segmentation image to obtain an edge image of each chamber;

[0041] a point cloud generation module, configured to generate corresponding point cloud data according to each of the edge images and the interpolation image;

[0042] The surface reconstruction module is used to perform surface reconstruction on the generated point cloud data through the Ashape algorithm to obtain a three-dimensional model.

[0043] In a third aspect, an embodiment of the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, the steps of the three-dimensional surface reconstruction method in the aforementioned first aspect or any implementation of the first aspect are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly describes the drawings required for the specific embodiments or the description of the prior art. Similar elements or parts are generally identified by similar reference numerals throughout the drawings. Elements or parts in the drawings are not necessarily drawn to scale.

[0045] FIG1 shows a flowchart of the steps of a method for three-dimensional surface reconstruction of a two-dimensional intracardiac ultrasound catheter image provided by an embodiment of the present invention;

[0046] FIG2 is a schematic diagram showing ultrasound image acquisition in step S1 of an embodiment of the present invention, wherein A is a schematic diagram showing acquisition of a continuous sequence of intracardiac ultrasound images, and B is a schematic diagram showing a single intracardiac ultrasound image;

[0047] FIG3 shows a schematic diagram of image interpolation in step S2 of an embodiment of the present invention, wherein A and B are two adjacent two-dimensional ultrasound images, and C is an interpolated image of A and B after linear interpolation;

[0048] FIG4 shows a schematic diagram of the Unet segmentation result in step S3 of an embodiment of the present invention, wherein A, B, and C are schematic diagrams of the left chamber of the heart at different angles;

[0049] FIG5 is a schematic diagram showing the edge extraction result in step S4 of an embodiment of the present invention, wherein A, B, and C are respectively the boundary contour information of the left chamber of the heart at different angles corresponding to FIG4 ;

[0050] FIG6 shows a schematic diagram of point cloud generation in step S5 of an embodiment of the present invention, wherein A is a schematic diagram of original point cloud data, and B is a schematic diagram after FPS sampling;

[0051] FIG7 shows a schematic diagram of the surface reconstruction effect of the Ashape algorithm in step S6 of an embodiment of the present invention;

[0052] FIG8 shows a schematic diagram of an ultrasonic device connection;

[0053] FIG9 shows a schematic structural diagram of an electronic device provided by an embodiment of the present invention;

[0054] FIG10 shows a structural block diagram of a three-dimensional surface reconstruction device for a two-dimensional intracardiac ultrasound catheter image provided by an embodiment of the present invention;

[0055] In FIG8 , 1. ultrasonic transducer, 2. ultrasonic catheter, 3. ultrasonic handle, 4. rotary motor, 5. ultrasonic connector, 6. ultrasonic main unit. DETAILED DESCRIPTION

[0056] The following embodiments of the technical solution of the present invention will be described in detail with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention and are therefore only examples and are not intended to limit the scope of protection of the present invention.

[0057] It should be noted that, unless otherwise specified, the technical or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0058] FIG1 is a flowchart of a method for three-dimensional surface reconstruction of a two-dimensional intracardiac ultrasound catheter image provided by an embodiment of the present invention. Referring to FIG1 , the method includes the following steps:

[0059] S1: Data acquisition: Multiple continuous ultrasound images are collected inside the heart through an intracardiac ultrasound catheter.

[0060] Intracardiac ultrasound catheter technology is used to acquire continuous ultrasound images. Specifically, the catheter is placed inside the heart and rotated along the ultrasound catheter head within a certain range of angles. As shown in Figure 2A, a continuous sequence of intracardiac ultrasound images is acquired. During this process, the motor rotates in steps, waiting for the sampling instructions sent by the electrocardiogram to perform sampling, while recording the rotation angle at each moment. During rotation, it can support real-time internal 360° high-speed rotation. To ensure that the acquired image sequence covers all angles and positions of the heart chamber and has good image quality, the sampling frequency during the acquisition process needs to be sampled according to the diastolic point of the patient's electrocardiogram to ensure that the three-dimensional model drawn after sampling is accurate. At the same time, changes in the entire process can be dynamically refreshed through real-time model reconstruction. The single acquisition result is shown in Figure 2B.

[0061] S2: Image interpolation: interpolation processing is performed on the blank areas between each ultrasound image to obtain multiple interpolated images.

[0062] For the acquired 2D ultrasound image data, since the motor rotation angle cannot meet the minimum modeling requirements, it is necessary to interpolate the blank areas between images. Edge images are interpolated to obtain interpolated images, which contain pixel information and angles to fill in any blank areas.

[0063] Interpolation must ensure not only image continuity but also overall image reliability, so linear interpolation is used to calculate pixel values. Linear interpolation uses the pixel values ​​of two adjacent points on the boundary of a known region and calculates the pixel value of the interpolated point based on the distance ratio. Suppose the point to be interpolated is P, and its adjacent known points are A and B. The distance between A and P is d1, and the distance between B and P is d2. The interpolated pixel value of point P can be calculated using the following formula:

[0064] P = A + (B - A) * (d1 / (d1 + d2)) (1)

[0065] This formula actually calculates the value of point P based on the linear relationship of line segment AB and the distance ratio.

[0066] FIG3 is an effect diagram of interpolation using the above method. A and B in FIG3 are two adjacent images obtained by ultrasound, and C in FIG3 is the result of linear interpolation from A and B in FIG3 .

[0067] S3: Target region segmentation: Each ultrasound image is input into the pre-trained Unet model to obtain a cardiac chamber segmentation image of each ultrasound image.

[0068] The Unet model is a commonly used convolutional neural network architecture, particularly suitable for pixel-level segmentation tasks. Its design is inspired by autoencoders. By connecting an encoder and a decoder, it can simultaneously learn local features and contextual information, thereby achieving accurate segmentation results.

[0069] The Unet model training method is as follows:

[0070] S3.1 Data preparation:

[0071] First, prepare a dataset of intracardiac ultrasound images with corresponding cardiac chamber annotations. Typically, these images contain various sections and views of the heart, as well as corresponding cardiac chamber boundary annotations.

[0072] S3.2 Data preprocessing:

[0073] Preprocess the intracardiac ultrasound images in the image dataset to remove noise, enhance contrast, and unify the image scale. Common preprocessing operations include filtering, histogram equalization, and normalization.

[0074] S3.3 Data division:

[0075] The preprocessed image dataset is divided into a training set, a validation set, and a test set. The training set is used to train the model, the validation set is used to adjust the model's hyperparameters and monitor the training progress, and the test set is used to evaluate the model's performance.

[0076] S3.4 Training Unet model:

[0077] The Unet model consists of an encoder and a decoder that can learn to perform pixel-level cardiac chamber segmentation. The encoder extracts image features, while the decoder remaps the features back to the input image dimensions and generates segmentation results. The Unet model is trained using intracardiac ultrasound image data and corresponding cardiac chamber annotations from a training set. A cross-entropy loss function is typically used to measure the difference between the predicted results and the true annotations, and the model parameters are optimized through backpropagation. Training can use gradient descent or its variants to minimize the loss function.

[0078] S3.5 verification and tuning:

[0079] Use the validation set to validate the UNet model during training to evaluate model performance and optimize it. You can select the best model based on the segmentation results and metrics (such as accuracy, recall, and F1 score) on the validation set.

[0080] S3.6 Testing and Evaluation:

[0081] The trained and validated UNet model is finally evaluated using the test set. By feeding the trained model intracardiac ultrasound images from the test set, the model generates cardiac chamber segmentation results. Various evaluation metrics, such as the Dice coefficient and Jaccard index, can be used to assess the model's performance and accuracy.

[0082] The heart chamber separation image output by Unet is shown in FIG4 , where the three images A, B, and C in FIG4 are the left chamber of the heart at different angles, respectively.

[0083] This step also includes:

[0084] Human-computer interaction: This part sets a manual annotation button in the interface for heart chamber images with annotation deviations to prevent machine errors.

[0085] The specific operation is that the result of automatic segmentation will be presented on the screen, and Unet will give a score to each image. The scoring is based on two main indicators: 1) Internal consistency, which evaluates the consistency and smoothness of the segmented image. The quality of the segmentation result can be evaluated by calculating the similarity or difference between adjacent pixels. For example, the difference in color, texture or gradient between pixels can be calculated, and the segmented image can be scored according to the size of the difference; 2) Regional connectivity: Evaluate whether the regions in the segmented image are connected. The area, perimeter or smoothness of the boundary of each region can be calculated, and different scores are given to the segmentation results based on these features. After weighted averaging the above evaluation indicators, they are arranged in ascending order. If they are lower than the set threshold, the corresponding user will be prompted to check the image. If they are inaccurate, manual intervention can be used to re-label the corresponding image.

[0086] S4: Edge extraction: extract edge information of each chamber surface of each cardiac chamber partition image to obtain an edge image of each chamber.

[0087] Edge extraction methods are used to obtain edge information on the cavity surface. Edge extraction captures the boundary contours between the cavity and other areas, allowing for further processing and analysis. Edge extraction helps determine the pixel locations of object boundaries, enabling accurate segmentation and reconstruction of the cavity area.

[0088] This embodiment uses a method based on the Sobel operator, which detects edges by calculating the gradient of pixel intensities in an image. This method is used to calculate the gradient of the image in the x and y directions. Assuming that the Sobel operator is used for each pixel position (x, y) in the input image, the magnitude G and direction θ of the image gradient can be calculated using the following formula:

[0089] (2)

[0090] (3)

[0091] Among them, Gx and Gy represent the gradients of the image in the x direction and y direction respectively. The larger the G value is, the more obvious the mutation of the pixel value near the edge will be, which means that there is edge information.

[0092] The cardiac chamber separation map output by Unet is used to calculate its boundary contour information using the Sobel operator, as shown in Figure 5.

[0093] S5: Point cloud generation: Generate corresponding point cloud data based on each edge image and interpolation image.

[0094] According to each edge image and the angle obtained during scanning (corresponding to the ultrasound image), the corresponding point cloud data is generated by combining the pixel information and angle of the interpolated image in step S2.

[0095] Assume that the original coordinates are the pixel position on the nth image (x, y), and its angle is θ. Assume that the transformed point cloud coordinates are , then the transformation relationship is shown in formula (4).

[0096] (4)

[0097] Where c is the offset constant.

[0098] The point cloud generated by the above method has too many points, which will increase the computational complexity of subsequent steps. In order to solve this problem, the farthest point sampling (FPS) is introduced to select a set of representative sampling points from large-scale point cloud data. The farthest point sampling method is based on the distance measurement between points. While maintaining a uniform distribution between sampling points, it covers the entire point cloud data as much as possible. In this embodiment, the farthest point sampling is used to maintain uniformity between data on the one hand, and to avoid the problem of too large a gap between triangles during subsequent surface reconstruction on the other hand. As shown in Figure 6, A in Figure 6 is the original point cloud data. The number of point clouds is large, which increases the computational complexity of surface reconstruction. After sampling FPS at 10,000 points, B in Figure 6 is obtained. Without losing the overall structure, the computational performance is improved.

[0099] S6: Surface reconstruction: The generated point cloud data is reconstructed using the Ashape algorithm to obtain a three-dimensional model.

[0100] The Ashape algorithm is a commonly used surface reconstruction algorithm that can restore a smooth cavity surface based on the geometric structure and topological relationships of point cloud data. Through the Ashape algorithm, the three-dimensional shape information of the cavity can be extracted from the point cloud data.

[0101] The generated point cloud data is reconstructed using the Ashape algorithm. The reconstructed chamber data is then visualized. 3D visualization software or libraries can be used to present the chamber reconstruction results, allowing doctors and researchers to more intuitively observe and analyze cardiac chamber structures, as shown in Figure 7.

[0102] It should be noted that the modules are arranged in a stream layout, which is only one embodiment of the present invention. Other arrangements may also be adopted, and the present invention does not limit this.

[0103] Unlike other intracardiac catheter image modeling methods, this 3D surface reconstruction method is based entirely on ultrasound images. It first acquires images of the cardiac cavity using an ultrasound catheter. Then, through interpolation and automatic segmentation, the cavity boundaries are determined. Three-dimensional coordinate mapping is performed on the edge-extracted image, and finally, surface reconstruction is performed using the AShape algorithm. This embodiment of the present invention has at least the following technical effects:

[0104] First, conventional cardiac cavity modeling currently relies on a second catheter, while this three-dimensional surface reconstruction method is an ultrasound image modeling method based on two-dimensional images and a single ultrasound probe, and based on the conversion of two-dimensional images into point clouds. This method reduces the operational process and difficulty, while supporting real-time updated cardiac chamber reconstruction, and can achieve automatic segmentation of cardiac cavity positions with faster speed and higher accuracy.

[0105] Second, in this 3D surface reconstruction method, the results of automatic segmentation are adjusted through human-computer interaction, and intervention is made at any time to ensure accuracy and reliability.

[0106] Third, in this 3D surface reconstruction method, after the cardiac chamber separation image is segmented, the boundary contour is extracted by edge and then converted into point cloud data, which greatly reduces the amount of calculation and improves efficiency.

[0107] This embodiment is based on intracardiac ultrasound catheter technology, while ultrasound clinical medical diagnostic technology is an ultrasound detection technology based on echo scanning. Ultrasonic diagnostic imaging uses a multi-element array transducer to transmit ultrasound waves into the human body. By changing the relative delay and amplitude of the excitation of each element, a focused sound beam emitted in a certain direction can be formed. When the sound beam encounters the interface between different organs and tissues in the body, a reflected echo is generated, which is then received by the array transducer. The above process requires the connection of multiple components. The device connection is shown in Figure 8. It specifically includes the following components:

[0108] Ultrasonic transducer 1, the ultrasonic transducer consists of 64 array elements, which complete the ultrasonic transmission and reception functions. It will be sent to the designated position of the heart for data collection.

[0109] Ultrasonic catheter 2, which contains various connecting wires of the sensor and flexible materials for bending.

[0110] Ultrasound handle 3, the doctor operates the handle to control the depth and direction of the catheter, which contains sensors such as a rotary motor.

[0111] The rotary motor 4 is connected to the catheter and can adjust the direction of the ultrasonic transducer.

[0112] Ultrasonic connector 5 realizes the conversion between analog signals and digital signals.

[0113] The ultrasound host 6 sends or receives instructions, obtains a two-dimensional image sequence through the ultrasound image acquisition device, and provides faster visual feedback to the doctor.

[0114] Figure 9 shows a schematic structural diagram of an electronic device 90 provided in an embodiment of the present invention, wherein the electronic device 90 includes at least one processor 901 (e.g., a CPU), at least one input / output interface 904, a memory 902, and at least one communication bus 903 for realizing connection and communication between these components. At least one processor 901 is used to execute computer instructions stored in the memory 902 so that the at least one processor 901 can execute any of the aforementioned embodiments of the three-dimensional surface reconstruction method. The memory 902 is a non-transitory memory, which may include a volatile memory, such as a high-speed random access memory (RAM), or a non-volatile memory, such as at least one disk storage. The communication connection with at least one other device or unit is realized through at least one input / output interface 904 (which may be a wired or wireless communication interface).

[0115] In some implementations, the memory 902 stores a program 9021 , and the processor 901 executes the program 9021 to execute the content of any of the aforementioned table partitioning method embodiments.

[0116] The electronic device may take many forms, including but not limited to:

[0117] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and their primary purpose is to provide voice and data communications. These terminals include smartphones, multimedia phones, feature phones, and low-end phones.

[0118] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, have computing and processing capabilities, and generally also have mobile Internet access. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0119] (3) Portable entertainment devices: These devices can display and play multimedia content. These devices include audio and video players (such as iPods), handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0120] (4) Specific server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0121] (5) Other electronic devices with data interaction functions.

[0122] FIG10 is a block diagram of a device for three-dimensional surface reconstruction of a two-dimensional intracardiac ultrasound catheter image provided by an embodiment of the present invention, the device comprising:

[0123] A data acquisition module, the data acquisition module is used to acquire multiple continuous ultrasound images inside the heart through an intracardiac ultrasound catheter;

[0124] An image interpolation module is used to perform interpolation processing on the blank areas between the ultrasound images to obtain multiple interpolated images;

[0125] A region segmentation module is used to input each ultrasound image into a pre-trained Unet model to obtain an output cardiac chamber segmentation image of each ultrasound image;

[0126] An edge extraction module is used to extract edge information of the surface of each chamber of each cardiac chamber separation image to obtain an edge image of each chamber;

[0127] A point cloud generation module is used to generate corresponding point cloud data according to each edge image and the interpolation image;

[0128] The surface reconstruction module is used to reconstruct the surface of the generated point cloud data through the Ashape algorithm to obtain a three-dimensional model.

[0129] The functions of each module in the embodiment of FIG10 correspond to the contents in the corresponding method embodiment and will not be repeated here.

[0130] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for three-dimensional surface reconstruction of a two-dimensional intracardiac ultrasound catheter image, characterized in that: The following steps are involved: S1: multiple continuous ultrasound images are collected inside the heart through an intracardiac ultrasound catheter; S2: performing interpolation processing on the blank areas between the ultrasound images to obtain a plurality of interpolation images; S3: inputting each of the ultrasound images into a pre-trained Unet model to obtain a cardiac chamber partition image of each of the ultrasound images output by the Unet model; S4: extracting edge information of the surface of each chamber of each cardiac chamber partition image to obtain an edge image of each chamber; S5: generating corresponding point cloud data according to each of the edge images and the interpolation image; S6: The generated point cloud data is reconstructed using the Ashape algorithm to obtain a three-dimensional model.

2. The three-dimensional surface reconstruction method according to claim 1, characterized in that: In the step S1, the ultrasound image includes rotation angle information corresponding to the intracardiac ultrasound catheter.

3. The three-dimensional surface reconstruction method according to claim 1, characterized in that: In the step S2, the interpolation process is linear interpolation, and the interpolation image includes pixel information and angle information.

4. The three-dimensional surface reconstruction method according to claim 1, characterized in that: In step S3, the Unet model training method is as follows: S3.1: Prepare a dataset of intracardiac ultrasound images with corresponding cardiac chamber annotations; S3.2: preprocessing the intracardiac ultrasound images in the intracardiac ultrasound image dataset to remove noise, enhance contrast, and unify the scale of the images; S3.3: dividing the preprocessed intracardiac ultrasound image dataset into a training set, a validation set, and a test set; S3.4: training the Unet model using the intracardiac ultrasound image data and the corresponding cardiac chamber annotations in the training set; S3.5: Using the validation set to validate the Unet model during training to evaluate the performance of the model and perform tuning; S3.6: Perform a final evaluation on the trained and validated Unet model using the test set.

5. The three-dimensional surface reconstruction method according to claim 4, characterized in that: In step S3.4, the cross entropy loss function is used to measure the difference between the predicted result and the true annotation, and the parameters of the model are optimized by back propagation. The training process uses a gradient descent algorithm to minimize the loss function.

6. The three-dimensional surface reconstruction method according to claim 4, characterized in that: The step S3 further includes: the Unet model determines each of the output cardiac chamber partition images, and if the determination is pending, manually performing secondary labeling on the corresponding cardiac chamber partition images.

7. The three-dimensional surface reconstruction method according to claim 1, characterized in that: In step S4, the edge information is extracted based on the Sobel operator in the following manner: For each pixel position in the cardiac chamber segmentation image , the magnitude of the gradient and direction It can be calculated by the following formula: ； ； in, and They represent the gradient of the pixels in the image in the x and y directions respectively.

8. The three-dimensional surface reconstruction method according to claim 1, characterized in that: In step S5, the point cloud data is generated as follows: ； in, are the point coordinates in the point cloud, (x, y) are the pixel positions in the edge image or the interpolation image, is the angle information corresponding to the edge image or the interpolation image, and c is the offset constant.

9. A three-dimensional surface reconstruction device for two-dimensional intracardiac ultrasound catheter images, characterized in that: include: A data acquisition module, the data acquisition module is used to acquire a plurality of continuous ultrasound images inside the heart through an intracardiac ultrasound catheter; An image interpolation module, the image interpolation module is used to perform interpolation processing on the blank areas between the ultrasound images to obtain a plurality of interpolated images; A region segmentation module, wherein the region segmentation module is used to input each of the ultrasound images into a pre-trained Unet model to obtain an output cardiac chamber segmentation image of each of the ultrasound images; An edge extraction module, the edge extraction module is used to extract edge information of each chamber surface of each of the cardiac chamber partition images to obtain an edge image of each of the chambers; A point cloud generation module, the point cloud generation module is used to generate corresponding point cloud data according to each of the edge images and the interpolation image; The surface reconstruction module is used to perform surface reconstruction on the generated point cloud data through the Ashape algorithm to obtain a three-dimensional model.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the three-dimensional surface reconstruction method according to any one of claims 1 to 8 are implemented.

Images