Right ventricle three-dimensional reconstruction method based on echocardiogram and related device

Abstract

The invention discloses a right ventricle three-dimensional reconstruction method based on an echocardiogram and a related device, and relates to the field of deep learning and three-dimensional reconstruction, and the method comprises the steps: obtaining the contour labeling data of the right ventricle of the heart in the echocardiogram; generating contour point voxel data and dense convex hull voxel data of three-dimensional contour points in a three-dimensional voxel space according to the contour labeling data; and inputting the contour point voxel data and the dense convex hull voxel data into the trained full convolutional neural network model to obtain a three-dimensional voxel model of the right ventricle of the heart. According to the method, the right ventricle three-dimensional model can be accurately and quickly constructed based on the echocardiography.

Description

Technical Field

[0001] The present application relates to the field of deep learning and three-dimensional reconstruction, and in particular to a method, device, equipment, medium and product for three-dimensional reconstruction of the right ventricle based on echocardiography. Background Art

[0002] Echocardiography has the advantages of high real-time performance, harmlessness to the human body, low cost, and high portability. It is the most common diagnostic method for cardiovascular diseases. Using echocardiography to reconstruct a three-dimensional model of the heart's ventricles can, on the one hand, help reproduce the three-dimensional model of the ventricles to assist surgery at a low cost and conveniently, and on the other hand, it can help detect ventricular diseases and cardiac function indicators. For example, it helps measure the most common parameter in assessing cardiac function: ejection fraction (EF). When calculating the EF value, it is necessary to measure the volume of the ventricles at the end of contraction and at the end of systole. However, due to the irregular shape of the right ventricle, its volume is usually difficult to measure directly. Therefore, using ultrasound images to reconstruct the contour of the right ventricle in three dimensions is of great significance for measuring three-dimensional indicators of cardiac function and reproducing three-dimensional models to assist surgery.

[0003] Echocardiographic data has the characteristics of parallel image sections and sparse data. Echocardiography is a cross-sectional image of the heart detected by a doctor holding a probe at different positions and angles in the patient's chest. Therefore, the cross-sectional image of an echocardiographic image is firstly non-parallel cross-sectional data with random geometric relationships. In addition, due to the limitations of conditions such as rib occlusion, the cross-sectional image of an echocardiographic image is generally sparse. Therefore, it is very difficult to reconstruct the right ventricle three-dimensionally based on the contour of the echocardiogram.

[0004] At present, the process of three-dimensional reconstruction of the right ventricle using echocardiography is mainly completed by knowledge-based methods (KBR). Doctors need to mark the coordinate values ​​of the key points representing the anatomical structures of the right ventricle on the echocardiogram, and then the KBR algorithm deforms a template model according to the key points of the anatomical structure to perform three-dimensional reconstruction. However, this process is time-consuming and laborious, requires the operator to have certain professional knowledge, and the accuracy is uncontrollable. At present, there is no automated method that can quickly and accurately perform three-dimensional reconstruction on ultrasound contour images.

[0005] The development of artificial intelligence is also driving the development of computer vision applications. Deep learning is used to solve the problem of 3D reconstruction. Currently, the most common method is to use deep learning models for multi-view 3D reconstruction. However, due to the characteristics of echocardiographic data, it is impossible to directly use multi-view 3D reconstruction models when using marker contours for 3D reconstruction on echocardiograms. Multi-view 3D reconstruction requires inputting images of an object at different perspectives to reconstruct the object in 3D. Obviously, the contour data of echocardiograms does not meet the input requirements of multi-view reconstruction. Therefore, in order to utilize the powerful 3D reconstruction capabilities of deep learning, it is necessary to design a new algorithm framework that adapts to the 3D reconstruction of the right ventricle using echocardiographic contours. Summary of the invention

[0006] The purpose of this application is to provide a three-dimensional reconstruction method of the right ventricle based on echocardiography and related devices, which can accurately and quickly construct a three-dimensional model of the right ventricle based on echocardiography.

[0007] To achieve the above objectives, this application provides the following solutions:

[0008] In a first aspect, the present application provides a method for three-dimensional reconstruction of the right ventricle based on echocardiography, comprising:

[0009] Acquire contour annotation data of the right ventricle of the heart in echocardiogram;

[0010] Generating contour point voxel data and dense convex hull voxel data of three-dimensional contour points in a three-dimensional voxel space according to the contour annotation data;

[0011] The contour point voxel data and the dense convex hull voxel data are input into the trained full convolutional neural network model to obtain a three-dimensional voxel model of the right ventricle of the heart.

[0012] In a second aspect, the present application provides a right ventricle three-dimensional reconstruction device based on echocardiography, comprising:

[0013] A data acquisition unit, used for acquiring contour annotation data of the right ventricle of the heart in the echocardiogram;

[0014] A three-dimensional voxel data generating unit, used for generating contour point voxel data and dense convex hull voxel data of three-dimensional contour points in a three-dimensional voxel space according to the contour annotation data;

[0015] The three-dimensional reconstruction unit is used to input the contour point voxel data and the dense convex hull voxel data into the trained full convolutional neural network model to obtain a three-dimensional voxel model of the right ventricle of the heart.

[0016] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the echocardiography-based three-dimensional reconstruction method of the right ventricle.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the echocardiography-based three-dimensional reconstruction method of the right ventricle.

[0018] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the echocardiography-based three-dimensional reconstruction method of the right ventricle.

[0019] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0020] The present application provides a three-dimensional reconstruction method and related device of the right ventricle based on echocardiography, which generates contour point voxel data and dense convex hull voxel data of three-dimensional contour points in three-dimensional voxel space according to contour annotation data, and inputs the contour point voxel data and dense convex hull voxel data into the trained full convolutional neural network model to obtain a three-dimensional voxel model of the right ventricle of the heart. Among them, dual-channel data sampling is used to construct voxel data according to the characteristics of echocardiography images, so that echocardiographic data with any number of sections and cross-sectional geometric relationships can be input into the full convolutional neural network model; for sparse echocardiographic data, the algorithm still maintains good robustness and can still ensure high algorithm accuracy. In practical applications, the three-dimensional reconstruction automation solution of the present invention has the advantages of being fast and accurate, can greatly reduce the workload and time of manual calculations, and can accurately and quickly construct a three-dimensional model of the right ventricle. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 This is a diagram of an application environment of a three-dimensional reconstruction method of a right ventricle based on echocardiography in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of a flow chart of a method for three-dimensional reconstruction of the right ventricle based on echocardiography provided in one embodiment of the present application;

[0024] Figure 3 A schematic diagram of an example structure of a fully convolutional neural network model provided in one embodiment of the present application;

[0025] Figure 4 A schematic diagram showing a comparison between the three-dimensional predicted structure of the right ventricle and the three-dimensional true structure of the right ventricle of the fully convolutional neural network model provided in one embodiment of the present application;

[0026] Figure 5 A schematic diagram of functional modules of a right ventricle three-dimensional reconstruction device based on echocardiography provided in one embodiment of the present application;

[0027] Figure 6 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0030] The right ventricle three-dimensional reconstruction method based on echocardiography provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store data that the server 104 needs to process. The data storage system can be set up separately, or it can be integrated on the server 104, or it can be placed on the cloud or other servers. The terminal 102 can send the contour annotation data of the right ventricle of the heart in the echocardiogram to the server 104. After the server 104 receives the contour annotation data of the right ventricle of the heart in the echocardiogram, for the contour annotation data of the right ventricle of the heart in the echocardiogram, the server 104 generates contour point voxel data and dense convex hull voxel data of the three-dimensional contour points in the three-dimensional voxel space according to the contour annotation data;

[0031] The contour point voxel data and the dense convex hull voxel data are input into the trained full convolutional neural network model to obtain a three-dimensional voxel model of the right ventricle of the heart. The server 104 can feed back the obtained three-dimensional voxel model of the right ventricle of the heart to the terminal 102. In addition, in some embodiments, the three-dimensional reconstruction method of the right ventricle based on the echocardiogram can also be implemented by the server 104 or the terminal 102 alone, such as the terminal 102 can directly perform three-dimensional reconstruction of the right ventricle of the heart based on the contour annotation data of the right ventricle of the heart in the echocardiogram, or the server 104 can obtain the contour annotation data of the right ventricle of the heart in the echocardiogram from the data storage system, and perform three-dimensional reconstruction of the right ventricle of the heart based on the contour annotation data of the right ventricle of the heart in the echocardiogram.

[0032] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0033] In an exemplary embodiment, Figure 2 As shown, a method for three-dimensional reconstruction of the right ventricle based on echocardiography is provided. The method is executed by a computer device, and specifically can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used for explanation, and the steps include the following steps 201 to 203.

[0034] Step 201, obtaining contour annotation data of the right ventricle of the heart in an echocardiogram.

[0035] The contour marking data of the echocardiogram includes the echocardiogram data of the ventricle and the surface contour of the ventricle marked in the echocardiogram data, wherein the ventricle contour is a ventricle contour line manually marked in the echocardiogram.

[0036] In this step, three-dimensional parameters of the echocardiography acquisition device may also be acquired, including 3D transformation parameters and calibration parameters.

[0037] Step 202: Generate contour point voxel data (corresponding to Figure 3 contours) and dense convex hull voxel data of 3D contour points (corresponding to Figure 3 The convex hull in the

[0038] Step 203: input the contour point voxel data and the dense convex hull voxel data into the trained full convolutional neural network model (corresponding to Figure 3 A three-dimensional voxel model of the right ventricle of the heart is obtained using the deep learning network in the image.

[0039] Implementing the above steps 201 to 203 provides a set of automated method frameworks for directly obtaining a three-dimensional model of the right ventricle based on cardiac ultrasound images. Unlike existing methods (such as the knowledge-based reconstruction method KBR), the present invention uses dual-channel data sampling and deep neural network technology to input echocardiographic data with any number of sections and cross-sectional geometric relationships; the algorithm still maintains good robustness for sparse echocardiographic data and can still ensure high algorithm accuracy. In practical applications, this automated solution has the advantages of being fast and accurate, can significantly reduce the workload and time of manual calculations, and greatly improves the efficiency of three-dimensional reconstruction.

[0040] In another exemplary embodiment of the present application, point sampling is first performed on the contour of the echocardiogram, and then the sampled two-dimensional points are transformed into a three-dimensional space matrix using the three-dimensional parameters of the ultrasound acquisition device to obtain a point set of these two-dimensional points in the three-dimensional space. That is, a set of points is sampled from the contour of the echocardiogram so that these points are evenly distributed on the contour. In this way, a coordinate set of two-dimensional points is obtained. Next, the coordinates of these two-dimensional points are dimensionally expanded and converted into a coordinate set of three-dimensional points. In order to map the coordinates of the two-dimensional points to the three-dimensional space, a three-dimensional transformation matrix is ​​generated using the information in the ultrasound acquisition device, and then the three-dimensional point coordinates are multiplied by the three-dimensional space matrix to obtain a three-dimensional coordinate set of the contour sampling points, that is, a three-dimensional contour sampling point set is obtained. In dual-channel processing, two channels are used to process the three-dimensional contour point data. The first channel performs point sampling on the three-dimensional contour points. Specifically, a voxel grid is created in a three-dimensional voxel space with the origin as the center, for example, the size of the grid is 128*128*128, and the side length of each voxel is a fixed value (for example, 0.016). Then, the three-dimensional contour sampling points are sampled in this voxel grid (the voxel value corresponding to the three-dimensional contour sampling point is 1, and the voxel value of the non-three-dimensional contour sampling point is 0), and a voxel data representing the distribution of the three-dimensional contour sampling points is obtained, that is, the contour point voxel data. At the same time, another channel solves the convex hull of the three-dimensional contour sampling points and performs dense convex hull sampling in the same three-dimensional voxel space (the value of the three-dimensional dense convex hull in the voxel space is represented as 1 (the voxel value of the convex hull surface and internal points is 1), and the voxel value of the non-dense convex hull point is 0). Specifically, the convex hull operation is performed on the three-dimensional contour sampling points (implemented by the existing convex hull algorithm, such as the quickhull convex hull algorithm) to obtain voxel data representing the convex hull shape of the three-dimensional contour sampling points, that is, dense convex hull voxel data, the size of which is the same as the size of the voxel grid, which is 128*128*128. Finally, by fusing the data of the two channels, the input data required for the neural network model in the subsequent steps is obtained, the size of which is 2*128*128*128. Therefore, in the present invention, step 202, generating contour point voxel data and dense convex hull voxel data of three-dimensional contour points in a three-dimensional voxel space according to the contour annotation data, specifically includes:

[0041] (1) Evenly distributed point sampling is performed on the right ventricle marker contour of the heart in the echocardiogram to obtain contour sampling points.

[0042] (2) Using the 3D transformation parameters and calibration parameters of the echocardiography data, the contour sampling points are subjected to a three-dimensional spatial matrix transformation to obtain three-dimensional contour sampling points.

[0043] In order to maintain consistency, the coordinates of the three-dimensional contour sampling points are normalized to ensure that their coordinate range is within a fixed range (-1 to 1). Therefore, after obtaining the three-dimensional contour sampling points, it also includes further normalizing the three-dimensional contour sampling points to obtain normalized three-dimensional contour sampling points, and recording the normalized scaling ratio.

[0044] (3) Representing the three-dimensional contour sampling points in a three-dimensional voxel space to obtain contour point voxel data.

[0045] (4) Determine a dense convex hull of the three-dimensional contour points according to the three-dimensional contour sampling points.

[0046] (5) Voxelizing the dense convex hull of the three-dimensional contour points in the three-dimensional voxel space to obtain the dense convex hull voxel data.

[0047] In another exemplary embodiment of the present application, the fully convolutional neural network model designed by the present invention has an encoder-decoder structure, and the encoder and the decoder together form a U-shaped network of an encoding and decoding structure. The encoder is responsible for feature extraction and downsampling of the input data; the decoder is responsible for restoring the feature map extracted by the encoder to the original resolution and generating the final three-dimensional voxel reconstruction result. The encoder and the decoder are connected through skip connections for feature fusion. The input three-dimensional data size of the fully convolutional neural network model is C*D*H*W voxel data, where C represents the number of channels, D represents depth, H represents height, and W represents width; the decoder in the fully convolutional neural network model outputs voxel data of size D*H*W.

[0048] like Figure 3 As shown, the fully convolutional neural network model includes an encoder and a decoder.

[0049] The encoder includes an input convolution module, a plurality of encoding convolution modules and a plurality of down-sampling modules; the decoder includes a plurality of decoding convolution modules, a plurality of skip connection modules, a plurality of up-sampling modules and an output convolution module. Figure 3 In the example, four encoding convolution modules, four downsampling modules, four decoding convolution modules, four skip connection modules and four upsampling modules are used. Of course, other values ​​can be selected according to the requirements, and there is no limitation here.

[0050] According to the data flow direction from the encoder to the decoder, several encoding convolution modules are defined as a first encoding convolution module, ..., an nth encoding convolution module, ..., an Nth encoding convolution module, and several downsampling modules are defined as a first downsampling module, ..., an nth downsampling module, ..., an Nth downsampling module.

[0051] According to the reverse direction of the data flow from the encoder to the decoder, several decoding convolution modules are defined as a first decoding convolution module, ..., an nth decoding convolution module, ..., an Nth decoding convolution module, several jump connection modules are defined as a first jump connection module, ..., an nth jump connection module, ..., an Nth jump connection module, and several upsampling modules are defined as a first upsampling module, ..., an nth upsampling module, ..., an Nth upsampling module.

[0052] The input of the input convolution module is the contour point voxel data and the dense convex hull voxel data; the output of the input convolution module is connected to the input of the first encoding convolution module and the input of the first jump connection module.

[0053] The output of the first encoding convolution module is connected to the input of the first down-sampling module, and the output of the first down-sampling module is respectively connected to the input of the second encoding convolution module and the input of the second skip connection module.

[0054] The output of the nth encoding convolution module is connected to the input of the nth downsampling module, and the output of the nth downsampling module is respectively connected to the input of the n+1th encoding convolution module and the input of the n+1th jump connection module; n=2,...,N-1.

[0055] The output of the Nth encoding convolution module is connected to the input of the Nth down-sampling module; the output of the Nth down-sampling module is connected to the input of the Nth decoding convolution module.

[0056] The output of the Nth decoding convolution module is connected to the input of the Nth upsampling module, the output of the Nth upsampling module is connected to the input of the Nth skip connection module, and the output of the Nth skip connection module is connected to the input of the N-1th decoding convolution module.

[0057] The output of the nth decoding convolution module is connected to the input of the nth upsampling module, the output of the nth upsampling module is connected to the input of the nth skip connection module, and the output of the nth skip connection module is connected to the input of the n-1th decoding convolution module.

[0058] The output of the first decoding convolution module is connected to the input of the first upsampling module, the output of the first upsampling module is connected to the input of the first jump connection module, the output of the first jump connection module is connected to the input of the output convolution module, and the output of the output convolution module is a three-dimensional voxel model of the right ventricle of the heart.

[0059] In another exemplary embodiment of the present application, Figure 3 As shown, after executing step 203 "inputting the contour point voxel data and the dense convex hull voxel data into the trained full convolutional neural network model to obtain a three-dimensional voxel model of the right ventricle of the heart", it also includes:

[0060] The three-dimensional voxel model of the right ventricle of the heart is restored according to the normalized scaling ratio; the volume of the restored three-dimensional voxel model is calculated; and the ejection fraction is calculated according to the end-diastolic and end-systolic volumes of the restored three-dimensional voxel model.

[0061] A simple proportional reduction post-processing operation is performed on the predicted results to obtain the volume of the corresponding right ventricle in a specific cycle. Finally, the ventricular volume results at the end of diastole and end of systole of the right ventricle are selected to calculate the ejection fraction index.

[0062] In another exemplary embodiment of the present application, before executing step 203 "inputting the contour point voxel data and the dense convex hull voxel data into the trained full convolutional neural network model", it also includes: training the full convolutional neural network model, and configuring a suitable software and hardware environment to train the full convolutional neural network model. During training, the optimizer uses Adam for iteration, the initial learning rate is 1e-6, the learning rate adjustment strategy adopts the cosine annealing strategy, and the batchsize is 24. A hybrid function of the two-category cross entropy and the Dice coefficient is used as the loss function for training.

[0063] The specific training process is:

[0064] (a) Obtain sample data of the contour annotation of the right ventricle of the heart in echocardiography.

[0065] Acquire multiple echocardiographic images of different cardiac cycles of cardiac patients and corresponding right ventricular contour annotations, 3D transformation parameters of echocardiographic data and calibration parameters.

[0066] (b) generating contour point voxel sample data and dense convex hull voxel sample data of three-dimensional contour points in a three-dimensional voxel space according to the contour annotation sample data.

[0067] (c) Inputting the contour point voxel sample data and the dense convex hull voxel sample data into the fully convolutional neural network model until the loss error of the model converges or the number of training iterations reaches a preset number of iterations, thereby obtaining the trained fully convolutional neural network model.

[0068] After the training is completed, the test set is used to further verify the model effect. The contour point voxel sample data and dense convex hull voxel sample data in the test set are input into the fully convolutional neural network model, and the three-dimensional voxel model of the right ventricle predicted by the test sample is obtained. After restoration according to the normalized scaling rate, it is compared with the real three-dimensional model of the right ventricle. The comparison results are shown in the figure. Figure 4 As shown, it can be seen that the present invention can accurately and efficiently predict the three-dimensional model of the right ventricle based on the echocardiographic contour.

[0069] In this embodiment, dual-channel data sampling is used to construct voxel data based on the characteristics of echocardiogram images, and then prediction is performed in three-dimensional voxel space through a deep learning network, and finally a right ventricular model of three-dimensional data is obtained. First, dual-channel point sampling and dense convex hull sampling voxel data are generated as input of the neural network based on the echocardiogram image with the right ventricular contour annotated; then, a full convolutional neural network based on encoding-decoding is used to reconstruct the three-dimensional right ventricular model; then, a series of post-processing measurement operations are performed on the prediction results, and finally the right ventricular three-dimensional model and three-dimensional index data of cardiac function are obtained.

[0070] The present application also provides an application scenario, which applies the above-mentioned three-dimensional reconstruction method of the right ventricle based on echocardiography. Specifically: the three-dimensional reconstruction method of the right ventricle based on echocardiography provided in this embodiment can be applied in the three-dimensional reconstruction scenario of the right ventricle. The three-dimensional reconstruction scene of the right ventricle includes a data acquisition link and a three-dimensional reconstruction link; the contour annotation data of the echocardiogram enters the three-dimensional reconstruction link from the data acquisition link. In the three-dimensional reconstruction method of the right ventricle based on echocardiography provided in this embodiment, obtaining the contour annotation data of the right ventricle of the heart in the echocardiogram belongs to the data acquisition link, and generating contour point voxel data and dense convex hull voxel data of the three-dimensional contour points in the three-dimensional voxel space according to the contour annotation data, and inputting the contour point voxel data and the dense convex hull voxel data into the trained full convolutional neural network model to obtain a three-dimensional voxel model of the right ventricle of the heart belongs to the three-dimensional reconstruction link.

[0071] Based on the same inventive concept, the embodiment of the present application also provides a device for implementing the above-mentioned three-dimensional reconstruction of the right ventricle based on echocardiography. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations of one or more embodiments of the three-dimensional reconstruction device of the right ventricle based on echocardiography provided below can refer to the limitations of the three-dimensional reconstruction method of the right ventricle based on echocardiography above, and will not be repeated here.

[0072] In an exemplary embodiment, Figure 5 As shown, a right ventricle three-dimensional reconstruction device based on echocardiography is provided, comprising:

[0073] The data acquisition unit M1 is used to acquire the contour annotation data of the right ventricle of the heart in the echocardiogram.

[0074] The three-dimensional voxel data generating unit M2 is used to generate contour point voxel data and dense convex hull voxel data of three-dimensional contour points in the three-dimensional voxel space according to the contour annotation data.

[0075] The three-dimensional reconstruction unit M3 is used to input the contour point voxel data and the dense convex hull voxel data into the trained full convolutional neural network model to obtain a three-dimensional voxel model of the right ventricle of the heart.

[0076] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video tag processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a three-dimensional reconstruction method of the right ventricle based on echocardiography is implemented.

[0077] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0078] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0079] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0080] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0081] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0082] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0083] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0084] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0085] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A three-dimensional reconstruction method of right ventricle based on echocardiography, characterized in that: The right ventricle three-dimensional reconstruction method based on echocardiography comprises: Acquire contour annotation data of the right ventricle of the heart in echocardiogram; Generating contour point voxel data and dense convex hull voxel data of three-dimensional contour points in a three-dimensional voxel space according to the contour annotation data; The contour point voxel data and the dense convex hull voxel data are input into the trained full convolutional neural network model to obtain a three-dimensional voxel model of the right ventricle of the heart.

2. The echocardiography-based three-dimensional reconstruction method of right ventricle according to claim 1, characterized in that: Generating contour point voxel data and dense convex hull voxel data of three-dimensional contour points in a three-dimensional voxel space according to the contour annotation data specifically includes: Evenly distributed point sampling is performed on the right ventricle marker contour of the heart in the echocardiogram to obtain contour sampling points; Performing a three-dimensional spatial matrix transformation on the contour sampling points using 3D transformation parameters and calibration parameters of the echocardiography data to obtain three-dimensional contour sampling points; Representing the three-dimensional contour sampling points in a three-dimensional voxel space to obtain contour point voxel data; Determine a dense convex hull of the three-dimensional contour points according to the three-dimensional contour sampling points; The dense convex hull of the three-dimensional contour points is voxelized in the three-dimensional voxel space to obtain the dense convex hull voxel data.

3. The method for three-dimensional reconstruction of the right ventricle based on echocardiography according to claim 2, characterized in that: Before executing the step of "representing the three-dimensional contour sampling points in a three-dimensional voxel space to obtain contour point voxel data", the echocardiogram-based right ventricle three-dimensional reconstruction method further includes: The three-dimensional contour sampling points are normalized to obtain normalized three-dimensional contour sampling points, and the normalized scaling ratio is recorded.

4. The method for three-dimensional reconstruction of the right ventricle based on echocardiography according to claim 1, characterized in that: The fully convolutional neural network model includes an encoder and a decoder; The encoder includes an input convolution module, a plurality of encoding convolution modules and a plurality of down-sampling modules; the decoder includes a plurality of decoding convolution modules, a plurality of skip connection modules, a plurality of up-sampling modules and an output convolution module; According to the data flow direction from the encoder to the decoder, a plurality of encoding convolution modules are defined as a first encoding convolution module, ..., an nth encoding convolution module, ..., and an Nth encoding convolution module, and a plurality of downsampling modules are defined as a first downsampling module, ..., an nth downsampling module, ..., and an Nth downsampling module; According to the reverse direction of the data flow from the encoder to the decoder, a plurality of decoding convolution modules are defined as a first decoding convolution module, ..., an nth decoding convolution module, ..., and an Nth decoding convolution module; a plurality of jump connection modules are defined as a first jump connection module, ..., an nth jump connection module, ..., and an Nth jump connection module; and a plurality of upsampling modules are defined as a first upsampling module, ..., an nth upsampling module, ..., and an Nth upsampling module; The input of the input convolution module is the contour point voxel data and the dense convex hull voxel data; the output of the input convolution module is connected to the input of the first encoding convolution module and the input of the first jump connection module; The output of the first encoding convolution module is connected to the input of the first down-sampling module, and the output of the first down-sampling module is connected to the input of the second encoding convolution module and the input of the second skip connection module respectively; The output of the nth encoding convolution module is connected to the input of the nth downsampling module, and the output of the nth downsampling module is respectively connected to the input of the n+1th encoding convolution module and the input of the n+1th skip connection module; n=2, ..., N-1; The output of the Nth encoding convolution module is connected to the input of the Nth downsampling module; the output of the Nth downsampling module is connected to the input of the Nth decoding convolution module; The output of the Nth decoding convolution module is connected to the input of the Nth upsampling module, the output of the Nth upsampling module is connected to the input of the Nth skip connection module, and the output of the Nth skip connection module is connected to the input of the N-1th decoding convolution module; The output of the nth decoding convolution module is connected to the input of the nth upsampling module, the output of the nth upsampling module is connected to the input of the nth skip connection module, and the output of the nth skip connection module is connected to the input of the n-1th decoding convolution module; The output of the first decoding convolution module is connected to the input of the first upsampling module, the output of the first upsampling module is connected to the input of the first jump connection module, the output of the first jump connection module is connected to the input of the output convolution module, and the output of the output convolution module is a three-dimensional voxel model of the right ventricle of the heart.

5. The method for three-dimensional reconstruction of the right ventricle based on echocardiography according to claim 3, characterized in that: After executing the step of "inputting the contour point voxel data and the dense convex hull voxel data into the trained full convolutional neural network model to obtain a three-dimensional voxel model of the right ventricle of the heart", the echocardiography-based three-dimensional reconstruction method of the right ventricle also includes: Restoring the three-dimensional voxel model of the right ventricle of the heart according to the normalized scaling ratio; Calculate the volume of the restored three-dimensional voxel model; The ejection fraction is calculated based on the end-diastolic and end-systolic volumes of the restored three-dimensional voxel model.

6. The method for three-dimensional reconstruction of right ventricle based on echocardiography according to claim 1, characterized in that: Before executing the step of "inputting the contour point voxel data and the dense convex hull voxel data into the trained full convolutional neural network model", it also includes: training the full convolutional neural network model, specifically including: Obtain sample data of contour annotation of the right ventricle of the heart in an echocardiogram; Generate contour point voxel sample data and dense convex hull voxel sample data of three-dimensional contour points in a three-dimensional voxel space according to the contour annotation sample data; The contour point voxel sample data and the dense convex hull voxel sample data are input into the full convolutional neural network model until the loss error of the model converges or the number of training iterations reaches a preset number of iterations, thereby obtaining the trained full convolutional neural network model.

7. A right ventricle three-dimensional reconstruction device based on echocardiography, characterized in that: The right ventricle three-dimensional reconstruction device based on echocardiography comprises: A data acquisition unit, used for acquiring contour annotation data of the right ventricle of the heart in the echocardiogram; A three-dimensional voxel data generating unit, used for generating contour point voxel data and dense convex hull voxel data of three-dimensional contour points in a three-dimensional voxel space according to the contour annotation data; The three-dimensional reconstruction unit is used to input the contour point voxel data and the dense convex hull voxel data into the trained full convolutional neural network model to obtain a three-dimensional voxel model of the right ventricle of the heart.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the echocardiography-based three-dimensional reconstruction method of the right ventricle as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for three-dimensional reconstruction of the right ventricle based on echocardiography described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for three-dimensional reconstruction of the right ventricle based on echocardiography described in any one of claims 1 to 6 is implemented.

Images