Three-dimensional reconstruction method and device for echocardiogram

Abstract

The invention provides an echocardiogram three-dimensional reconstruction method and device, and relates to the technical field of medical image.The echocardiogram three-dimensional reconstruction method comprises the steps that a target preprocessing strategy, a target segmentation strategy and a target reconstruction strategy are determined based on residual computing power resources of an ultrasonic workstation; for the obtained multiple frames of cardiac ultrasound images of the tested object, based on a target preprocessing strategy, the multiple frames of cardiac ultrasound images are preprocessed, and multiple frames of target ultrasound images are obtained; based on the target segmentation strategy, performing image segmentation processing on the target ultrasonic image to obtain a semantic tag graph; and based on a target image reconstruction strategy, performing three-dimensional reconstruction on the plurality of semantic tag maps to obtain a three-dimensional image model of the heart of the measured object.

Description

Technical Field

[0001] The present disclosure relates to the field of medical imaging technology, and more particularly, to a method and apparatus for three-dimensional reconstruction of echocardiography. Figure 3 Background Art

[0002] Echocardiography is a diagnostic tool that uses ultrasonic imaging technology to observe and evaluate the structure and function of the heart. An ultrasonic probe is placed on the chest, and ultrasonic signals emitted and received by the ultrasonic probe are used to generate images of the heart. Due to the characteristics of safety, convenience, non-invasiveness, and economy, echocardiography has become an indispensable auxiliary means in the diagnosis and treatment of heart diseases.

[0003] In the process of implementing the concept of the present disclosure, the inventors found that there are at least the following problems in the related art: It is urgent to further improve echocardiography technology to improve the image resolution, enhance real-time performance, and other performances of echocardiography. Summary of the Invention

[0004] In view of this, the present disclosure provides a method and apparatus for three-dimensional reconstruction of echocardiography, which can improve the image resolution of echocardiography and enhance real-time performance. Figure 3

[0005] One aspect of the present disclosure provides a method for three-dimensional reconstruction of echocardiography, including: respectively determining a target preprocessing strategy, a target segmentation strategy, and a target reconstruction strategy based on the remaining computing power resources of an ultrasonic workstation; for multiple frames of cardiac ultrasonic images of a measured object obtained, preprocessing the multiple frames of cardiac ultrasonic images respectively based on the target preprocessing strategy to obtain multiple frames of target ultrasonic images; performing image segmentation processing on the target ultrasonic images based on the target segmentation strategy to obtain a semantic label map; and performing three-dimensional reconstruction on multiple semantic label maps based on the target image reconstruction strategy to obtain a three-dimensional image model of the heart of the measured object. Figure 3

[0006] According to an embodiment of the present disclosure, performing image segmentation processing on the target ultrasonic images based on the target segmentation strategy to obtain a semantic label map includes: when the target segmentation strategy is represented as a high-performance processing strategy, using N encoders of an encoder-decoder network to process the target ultrasonic images to obtain N encoded features, N>1; using N decoders of the encoder-decoder network to process the N encoded features respectively to obtain N decoded features; splicing the N decoded features to obtain a fused feature; and inputting the fused feature into a convolutional network to obtain the semantic label map.

[0007] According to an embodiment of the present disclosure, processing the target ultrasound image by N encoders of the encoder-decoder network to obtain N encoded features includes: for the nth encoder, segmenting and linearly mapping the encoded features output by the (n - 1)th encoder to obtain a feature sequence, where n ≥ 1, and the encoded features output by the 0th encoder are represented as the target ultrasound image; processing the feature sequence by M network blocks of the nth encoder to obtain a target output sequence, where M ≥ 1; and rearranging the target output sequence to obtain the encoded features output by the nth encoder.

[0008] According to an embodiment of the present disclosure, processing the feature sequence by M network blocks of the nth encoder to obtain a target output sequence includes: for the mth network block, normalizing the input sequence of the mth network block to obtain a normalized sequence, where m ≥ 1, and the input sequence of the 1st network block is represented as the feature sequence; linearly projecting the normalized sequence to obtain a first projection sequence and a second projection sequence; processing the first projection sequence by a forward convolutional layer and a forward state space layer to obtain a forward sequence; processing the first projection sequence by a backward convolutional layer and a backward state space layer to obtain a backward sequence; performing gating and merging processing on the forward sequence and the backward sequence based on the second projection sequence to obtain a first output sequence; sequentially performing downsampling processing, non-linear activation processing, and upsampling processing on the input sequence of the mth network block to obtain a second output sequence; and merging the first output sequence and the second output sequence to obtain the output sequence of the mth network block, where the output sequence of the Mth network block is represented as the target output sequence.

[0009] According to an embodiment of the present disclosure, processing the N encoded features by N decoders of the encoder-decoder network to obtain N decoded features includes: for the nth decoder, performing bilinear interpolation on the nth encoded feature by the nth decoder to obtain the nth decoded feature; where n ≥ 1.

[0010] According to an embodiment of the present disclosure, performing image segmentation processing on the target ultrasound image based on the target segmentation strategy to obtain a semantic label map includes: in the case where the target segmentation strategy is represented as a low-performance processing strategy, performing threshold segmentation processing on the target ultrasound image to obtain the semantic label map.

[0011] According to an embodiment of the present disclosure, based on the above-mentioned target image reconstruction strategy, three-dimensional reconstruction is performed on multiple semantic label maps to obtain a three-dimensional image model of the heart of the object to be measured, including: when the target image reconstruction strategy is represented as a low-performance processing strategy, surface rendering is performed on multiple semantic label maps to obtain the three-dimensional image model; and when the target image reconstruction strategy is represented as a high-performance processing strategy, volume rendering is performed on multiple semantic label maps to obtain the three-dimensional image model.

[0012] According to an embodiment of the present disclosure, based on the above-mentioned target preprocessing strategy, multiple frames of cardiac ultrasound images are preprocessed respectively to obtain multiple frames of target ultrasound images, including: when the target preprocessing strategy is represented as a low-performance processing strategy, adaptive histogram equalization processing is performed on the cardiac ultrasound images to obtain the target ultrasound images; and when the target preprocessing strategy is represented as a high-performance processing strategy, a denoising network is used to suppress noise in the cardiac ultrasound images to obtain the target ultrasound images.

[0013] According to an embodiment of the present disclosure, the ultrasound workstation is configured with an ultrasound probe, and the method further includes: when the ultrasound probe is set in the detection area of the object to be measured, in response to a detection instruction, controlling the ultrasound probe to collect the multiple frames of cardiac ultrasound images.

[0014] Another aspect of the present disclosure provides a three-dimensional reconstruction device for echocardiography Figure 3 including: a determination module for respectively determining a target preprocessing strategy, a target segmentation strategy, and a target reconstruction strategy based on the remaining computing power resources of the ultrasound workstation; a preprocessing module for, for multiple frames of cardiac ultrasound images of the object to be measured obtained, preprocessing the multiple frames of cardiac ultrasound images respectively based on the target preprocessing strategy to obtain multiple frames of target ultrasound images; an image segmentation module for performing image segmentation processing on the target ultrasound images based on the target segmentation strategy to obtain semantic label maps; and a three-dimensional reconstruction module for performing three-dimensional reconstruction on multiple semantic label maps based on the target image reconstruction strategy to obtain a three-dimensional image model of the heart of the object to be measured

[0015] According to an embodiment of the present disclosure, by preprocessing multiple frames of cardiac ultrasound images, the quality of the ultrasound image data is ensured; performing image segmentation processing on the target ultrasound images can improve the accuracy of the ultrasound images and enable the three-dimensional reconstruction results to be more accurate and reliable. The three-dimensional reconstruction device for echocardiography according to the embodiments of the present disclosure Figure 3The 3D reconstruction method, for different working scenarios, divides the computing power performance based on the remaining computing power resources into low performance and high performance, determines the corresponding processing strategies, dynamically adjusts the algorithms and parameter settings in each link, realizes the full-process optimization of the preprocessing, segmentation, and 3D reconstruction of echocardiograms, ensures the processing efficiency and result quality under different computing power conditions, has good computing power adaptability, and improves the performance of echocardiogram image resolution and real-time enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Through the following description of the embodiments of the present disclosure with reference to the drawings, the above and other objects, features, and advantages of the present disclosure will become clearer. In the drawings:

[0017] Figure 1 Schematically shows an exemplary system architecture 100 to which the echocardiogram Figure 3 3D reconstruction method can be applied;

[0018] Figure 2 Schematically shows an operation flowchart of the echocardiogram Figure 3 3D reconstruction method according to an embodiment of the present disclosure;

[0019] Figure 3 Schematically shows a schematic diagram of the enhancement effect of adaptive histogram equalization processing according to an embodiment of the present disclosure;

[0020] Figure 4 Schematically shows a schematic diagram of obtaining a semantic label map of the echocardiogram Figure 3 3D reconstruction method according to an embodiment of the present disclosure;

[0021] Figure 5 Schematically shows a schematic diagram of obtaining N encoded features of the echocardiogram Figure 3 3D reconstruction method according to an embodiment of the present disclosure;

[0022] Figure 6 Schematically shows a schematic diagram of obtaining a target output sequence of the echocardiogram Figure 3 3D reconstruction method according to an embodiment of the present disclosure;

[0023] Figure 7 Schematically shows a schematic diagram of a chest model of 3D reconstruction of the echocardiogram Figure 3 3D reconstruction method according to an embodiment of the present disclosure;

[0024] Figure 8 Schematically shows a block diagram of an echocardiogram Figure 3 3D reconstruction device according to an embodiment of the present disclosure; and

[0025] Figure 9 Schematically shows a schematic diagram suitable for implementing the echocardiogramFigure 3 Block diagram of an electronic device for a three-dimensional reconstruction method. Detailed implementation manners

[0026] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for the sake of explanation, numerous specific details are set forth in order to provide a comprehensive understanding of the embodiments of the present disclosure. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present disclosure.

[0027] The terms used herein are merely for describing specific embodiments and are not intended to limit the present disclosure. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0028] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0029] In the case of using expressions such as "at least one of A, B, and C", generally, it should be interpreted according to the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include, but is not limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0030] As a non-invasive and convenient medical imaging technology, echocardiography has been widely used in the field of clinical diagnosis and evaluation of heart diseases. Echocardiography can effectively detect and evaluate various heart diseases such as cardiac structural abnormalities, heart valve diseases, myocardial lesions, and heart failure. During cardiac surgery, real-time echocardiography can provide key imaging information, thus facilitating the smooth progress of the surgery and the accurate formulation of intraoperative decisions. For example, during coronary artery bypass grafting surgery, echocardiography can enable medical staff to accurately locate blood vessels and effectively evaluate the postoperative surgical effect, providing strong support for the successful implementation of the surgery. For example, during valve replacement surgery, echocardiography can be used to accurately evaluate the functional status and position of the artificial valve to ensure that the surgery achieves the expected therapeutic effect.

[0031] Three-dimensional echocardiography technology provides more accurate and intuitive imaging support for heart surgery. Three-dimensional echocardiography can, through three-dimensional reconstruction technology, enable medical staff to view the anatomical structure and functional status of the heart from any angle, greatly enhancing the visualization level of intraoperative images. For example, in clinical applications, three-dimensional echocardiography can provide three-dimensional images of various cross-sections of the heart, enabling medical staff to comprehensively evaluate heart lesions and their relationships with surrounding tissues. For example, in postoperative functional evaluation, three-dimensional echocardiography, through the intuitive evaluation of the postoperative heart structure and function, enables medical staff to promptly detect and handle possible complications, thereby improving the quality of patients' postoperative recovery. It can also help medical staff judge the surgical effect and formulate corresponding rehabilitation plans. The application of three-dimensional echocardiography technology not only improves the success rate of heart surgery but also provides an important guarantee for patients' long-term health.

[0032] Therefore, it is urgent to further improve three-dimensional echocardiography technology to enhance performance such as the image resolution of echocardiography and real-time performance.

[0033] Embodiments of the present disclosure provide a method and device for echocardiographic Figure 3 three-dimensional reconstruction, which can enhance the image resolution of echocardiography and real-time performance. The echocardiographic Figure 3 three-dimensional reconstruction method includes: respectively determining a target preprocessing strategy, a target segmentation strategy, and a target reconstruction strategy based on the remaining computing power resources of an ultrasound workstation; for multiple frames of cardiac ultrasound images of a measured object obtained, preprocessing the multiple frames of cardiac ultrasound images respectively based on the target preprocessing strategy to obtain multiple frames of target ultrasound images; performing image segmentation processing on the target ultrasound images based on the target segmentation strategy to obtain a semantic label map; and performing three-dimensional reconstruction on multiple semantic label maps based on the target image reconstruction strategy to obtain a three-dimensional image model of the heart of the measured object.

[0034] Figure 1 Schematically shows an exemplary system architecture 100 to which the echocardiographic Figure 3 three-dimensional reconstruction method according to an embodiment of the present disclosure can be applied. It should be noted that Figure 1 only the shown is an example of the system architecture to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0035] As Figure 1As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0036] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).

[0037] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc.

[0038] The server 105 may be a server that provides various services, such as a background management server that supports the websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (only for example). The background management server may analyze and process data such as received user requests, etc., and feedback the processing results (such as web pages, information, or data, etc. obtained or generated according to user requests) to the terminal device.

[0039] It should be noted that the echocardiogram Figure 3 three-dimensional reconstruction method provided by the embodiments of the present disclosure can generally be executed by the server 105. Correspondingly, the echocardiogram Figure 3 three-dimensional reconstruction device provided by the embodiments of the present disclosure can generally be set in the server 105. The echocardiogram Figure 3 three-dimensional reconstruction method provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105. Correspondingly, the echocardiogram Figure 3 three-dimensional reconstruction device provided by the embodiments of the present disclosure can also be set in a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105. Or, the echocardiogram Figure 3The three-dimensional reconstruction method can also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or can also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the echocardiography Figure 3 The three-dimensional reconstruction device can also be disposed in the first terminal device 101, the second terminal device 102, or the third terminal device 103, or disposed in other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0040] For example, the image to be processed can be originally stored in any one of the first terminal device 101, the second terminal device 102, or the third terminal device 103 (for example, the first terminal device 101, but not limited thereto), or stored on an external storage device and can be imported into the first terminal device 101. Then, the first terminal device 101 can execute the image processing method provided by the embodiments of the present disclosure locally, or send the image to be processed to other terminal devices, a server, or a server cluster, and the other terminal devices, server, or server cluster that receives the image to be processed execute the echocardiography Figure 3 three-dimensional reconstruction method.

[0041] It should be understood that Figure 1 the numbers of terminal devices, networks, and servers in

[0042] Figure 2 are merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers. Figure 3 The flowchart of the echocardiography

[0043] As Figure 2 shown, the echocardiography Figure 3 three-dimensional reconstruction method includes operations S210 to S240.

[0044] In operation S210, based on the remaining computing power resources of the ultrasound workstation, a target preprocessing strategy, a target segmentation strategy, and a target reconstruction strategy are respectively determined.

[0045] In operation S220, for multiple frames of cardiac ultrasound images of the measured object obtained, based on the target preprocessing strategy, the multiple frames of cardiac ultrasound images are respectively preprocessed to obtain multiple frames of target ultrasound images.

[0046] In operation S230, based on the target segmentation strategy, image segmentation processing is performed on the target ultrasound images to obtain a semantic label map.

[0047] In operation S240, based on the target image reconstruction strategy, three-dimensional reconstruction is performed on multiple semantic label maps to obtain a three-dimensional image model of the heart of the object under test.

[0048] According to the embodiments of the present disclosure, by preprocessing multiple frames of cardiac ultrasound images, the quality of the ultrasound image data is ensured; image segmentation processing of the target ultrasound image can improve the accuracy of the ultrasound image, enabling the three-dimensional reconstruction result to be more accurate and reliable. The echocardiography Figure 3 three-dimensional reconstruction method of the present disclosure embodiment is oriented to different working scenarios, and is divided into low performance and high performance based on the computing power performance of the remaining computing power resources, determines the corresponding processing strategy, dynamically adjusts the algorithms and parameter settings of each link, and realizes the full-process optimization of the preprocessing, segmentation, and three-dimensional reconstruction of echocardiograms, ensuring the processing efficiency and result quality under different computing power conditions, having good computing power adaptability, and improving the image resolution and real-time performance of echocardiograms.

[0049] In one example, the remaining computing power resources of the ultrasound workstation refer to the remaining computing power resources of the hardware configuration of the ultrasound workstation. For example, the CPU, the number of cores of the CPU, the main frequency, the memory, and the video memory capacity of the ultrasound workstation, etc. Among them, the CPU represents the central processing unit; the number of cores of the CPU represents the number of independent processing units inside the CPU; the main frequency represents the clock frequency of the CPU, reflecting the speed of the CPU to process data; the video memory capacity represents the dedicated memory of the graphics processing unit, which is used to store graphic data. The performance indicators for evaluating the remaining computing power resources of the ultrasound workstation can be CPU usage, GPU usage, memory usage, disk usage, network bandwidth usage, and task queue length, etc. The performance indicators of the remaining computing power resources of the ultrasound workstation can be collected through system monitoring tools or programming interfaces. According to the actual application scenario and requirements, the thresholds for high performance and low performance of the computing power performance are defined. For example, CPU usage: below 30% is high performance, above 80% is low performance. For example, GPU usage: below 20% is high performance, above 90% is low performance. For example: if the performance of the CPU, GPU, and memory are all high performance, then the system as a whole is high performance. If any one of the key resources (such as the CPU or GPU) is low performance, then the system as a whole is low performance.

[0050] In one example, multiple frames of cardiac ultrasound images of the object under test are obtained from the ultrasound workstation. Multiple frames of cardiac ultrasound images are usually stored in JPEG or DICOM (DCM) format. DICOM is a widely used medical image format that contains image data and related metadata (such as patient information, device parameters, etc.), and multiple frames of cardiac ultrasound images can be read by an image reader and its origin can be detected. The process of reading multiple frames of cardiac ultrasound images is as follows:

[0051] Set the number of scalar components of the multi-frame cardiac ultrasound images. The number of scalar components usually refers to the number of channels of the image. For ultrasound images, it is usually single-channel (grayscale image). When reading multi-frame cardiac ultrasound images, correctly set the number of channels to match the actual format of the multi-frame cardiac ultrasound images to ensure data accuracy when reading multi-frame cardiac ultrasound images.

[0052] Set the file dimensions and data range to match the actual size and data range of the multi-frame cardiac ultrasound images.

[0053] Set the file prefix and file mode to read the image files of multi-frame cardiac ultrasound images in a specific format.

[0054] Set the data byte order to ensure correct decoding of the data of the multi-frame cardiac ultrasound images.

[0055] Ultrasound images are usually affected by noise interference, which affects the quality of ultrasound images. Therefore, denoising and enhancement processing are required. After the multi-frame cardiac ultrasound images are read, medical ultrasound image preprocessing can be performed on the multi-frame cardiac ultrasound images respectively.

[0056] According to an embodiment of the present disclosure, when the target preprocessing strategy is represented as a low-performance processing strategy, perform adaptive histogram equalization processing on the cardiac ultrasound image to obtain a target ultrasound image.

[0057] In one example, a method of spatial domain processing can be adopted: Contrast Limited Adaptive Histogram Equalization (CLAHE) to perform preprocessing on the multi-frame cardiac ultrasound images respectively. Adaptive Histogram Equalization (AHE) processing is an effective image enhancement method, which can significantly improve the contrast and detail expressiveness of the image. CLAHE is an improved method of AHE, which reduces the amplification problem of noise by restricting the amplification degree of contrast.

[0058] Figure 3 A schematic diagram schematically shows the enhancement effect of the adaptive histogram equalization processing according to an embodiment of the present disclosure.

[0059] As Figure 3 shown, Figure 3 the left figure of Figure 3 is the cardiac ultrasound image before enhancement processing, and

[0060] the right figure of is the cardiac ultrasound image after enhancement processing. The enhancement processing improves the contrast of the cardiac ultrasound image.

[0060] According to an embodiment of the present disclosure, when the target preprocessing strategy is represented as a high-performance processing strategy, a denoising network is used to suppress noise in the cardiac ultrasound image to obtain the target ultrasound image.

[0061] In one example, Convolutional Neural Networks (CNN) are suitable for processing data with a grid structure. The denoising network can be DnCNN (Deep Neural Network for Image Denoising), FFDnet (Fast and Flexible Denoising Network), CBDnet (Convolutional Blind Denoising Network), etc. By using a pre-trained denoising network to separately identify and suppress the noise components in multiple frames of cardiac ultrasound images and retain the effective image information, the speckle noise and random noise in the multiple frames of cardiac ultrasound images can be effectively reduced to improve the clarity of the multiple frames of cardiac ultrasound images and obtain the target ultrasound image.

[0062] According to an embodiment of the present disclosure, when the target preprocessing strategy is represented as a low-performance processing strategy, enhancement processing improves the contrast and details of the target ultrasound image, enabling medical staff to more easily observe and analyze the image. When the target preprocessing strategy is represented as a high-performance processing strategy, processing by a deep learning denoising network reduces noise interference, making the target ultrasound image clearer. Preprocessing multiple frames of cardiac ultrasound images to reduce speckle noise and random noise can significantly improve the quality and accuracy of multiple frames of target ultrasound images.

[0063] When the target segmentation strategy is represented as a high-performance processing strategy, the target ultrasound image can be segmented by deep learning models such as U-Net, Fully Convolutional Network (FCN), SAM (Segment Anything Model), etc.

[0064] According to an embodiment of the present disclosure, when the target segmentation strategy is represented as a high-performance processing strategy, N encoders of an encoder-decoder network are used to process the target ultrasound image to obtain N encoded features, where N > 1; N decoders of the encoder-decoder network are respectively used to process the N encoded features to obtain N decoded features; the N decoded features are concatenated to obtain a fused feature; and the fused feature is input into a convolutional network to obtain a semantic label map.

[0065] According to an embodiment of the present disclosure, for the nth encoder, the encoded features output by the (n - 1)th encoder are segmented and linearly mapped to obtain a feature sequence, where n ≥ 1, and the encoded features output by the 0th encoder are represented as the target ultrasound image; the M network blocks of the nth encoder are used to process the feature sequence to obtain a target output sequence, where M ≥ 1; and the target output sequence is rearranged to obtain the encoded features output by the nth encoder.

[0066] According to an embodiment of the present disclosure, the segmentation and linear mapping of the encoded features output by the (n - 1)th encoder may be to segment the input encoded features into multiple small regions, and map each small region to a one-dimensional vector through a fully connected layer or a convolutional layer, that is, one input encoded feature will be mapped to multiple one-dimensional vectors.

[0067] Figure 4 Schematically shows an echocardiogram Figure 3 Schematic diagram of obtaining a semantic label map of a three-dimensional reconstruction method according to an embodiment of the present disclosure.

[0068] As Figure 4 shown, in one example, the first encoder of the encoder-decoder network is used to process the target ultrasound image (B, 3, 448, 448) to obtain the encoded features (B, 128, 112, 112) output by the first encoder, where (B, 3, 448, 448) is the original size of the input image, the target ultrasound image, B is the batch size, 3 is the number of channels of the image (RGB), and 488×488 is the resolution of the target ultrasound image; (B, 128, 112, 112) is the feature map after the first layer of convolution and pooling, 128 is the number of channels of this layer (i.e., the number of convolutional kernels), and 112×112 is the spatial resolution of the encoded features output by the first encoder, and the target ultrasound image has been downsampled by a factor of 2.

[0069] The second encoder is used to process the encoded features (B, 128, 112, 112) output by the first encoder to obtain the encoded features (B, 256, 56, 56) output by the second encoder. The third encoder is used to process the encoded features (B, 256, 56, 56) output by the second encoder to obtain the encoded features (B, 512, 28, 28) output by the third encoder. The fourth encoder is used to process the encoded features (B, 512, 28, 28) output by the third encoder to obtain the encoded features (B, 1024, 14, 14) output by the fourth encoder, and the target ultrasound image has been downsampled by a factor of 32.

[0070] Figure 5 Schematically shows an echocardiogram Figure 3 Schematic diagram of obtaining N encoded features of a three-dimensional reconstruction method according to an embodiment of the present disclosure.

[0071] As Figure 5 shown, in one example, the first encoder of the encoder-decoder network is used to segment and linearly map the target ultrasound image, and a feature sequence output by the first encoder can be obtained. The M image patches of the first encoder are used to process the feature sequence output by the first encoder, and a target output sequence output by the first encoder can be obtained. The target output sequence output by the first encoder is rearranged to obtain the encoded feature output by the first encoder.

[0072] As Figure 5 shown, in one example, when n = 4, n - 1 = 3, and N = 10, the fourth encoder of the encoder-decoder network is used to segment and linearly map the encoded feature output by the third encoder, and a feature sequence output by the fourth encoder can be obtained. The M image patches of the fourth encoder are used to process the feature sequence output by the fourth encoder, and a target output sequence output by the fourth encoder can be obtained. The target output sequence output by the fourth encoder is rearranged to obtain the encoded feature output by the fourth encoder. Similarly, the tenth encoder of the encoder-decoder network is used to process the encoded feature output by the ninth encoder to obtain the encoded feature output by the tenth encoder. The ten decoders of the encoder-decoder network are used to process the ten encoded features respectively to obtain ten decoded features.

[0073] Figure 6 Schematically shows a schematic diagram of obtaining a target output sequence of an ultrasonic echocardiogram Figure 3 3D reconstruction method according to an embodiment of the present disclosure.

[0074] As Figure 6 shown, according to an embodiment of the present disclosure, for the m-th network block, the input sequence of the m-th network block is normalized to obtain a normalized sequence, where m ≥ 1, and the input sequence of the first network block is represented as a feature sequence. The normalized sequence is linearly projected to obtain a first projection sequence (x) and a second projection sequence (z).

[0075] As Figure 6 shown, according to an embodiment of the present disclosure, the forward convolutional layer and the forward state space layer are used to process the first projection sequence to obtain a forward sequence. The backward convolutional layer and the backward state space layer are used to process the first projection sequence to obtain a backward sequence. Based on the second projection sequence, gating and merging processing are performed on the forward sequence and the backward sequence to obtain a first output sequence.

[0076] As Figure 6 shown, according to an embodiment of the present disclosure, the input sequence of the m-th network block is successively subjected to downsampling processing, non-linear activation processing, and upsampling processing to obtain a second output sequence.

[0077] As shown Figure 6 According to an embodiment of the present disclosure, the first output sequence and the second output sequence are combined to obtain the output sequence of the m-th network block, where the output sequence of the M-th network block is represented as the target output sequence.

[0078] According to an embodiment of the present disclosure, for example, when n = 5, m = 3, and M = 10, the input sequence of the first network block of the fifth encoder is the feature sequence output by the fourth encoder. The input sequence of the third network block of the fifth encoder is the output sequence of the second network block of the fifth encoder, and the output sequence of the tenth network block of the fifth encoder is the target output sequence of the fifth encoder.

[0079] According to an embodiment of the present disclosure, VIM (Vision Mamba Block) is a core component of a new type of vision foundation model, designed based on the Bidirectional State Space Model (SSM) for efficient processing of vision tasks. The network block is modified and designed with reference to VIM, adding downsampling to the side features. After adding downsampling, non-linear activation, and upsampling operations, the receptive field can be expanded to extract higher-level abstract features. Non-linear activation can introduce non-linear transformations to enhance the expressive power of the network. Using the U-Net model through the encoder-decoder structure, image features can be efficiently extracted and accurate pixel-level segmentation can be performed. By training this model, tissues or organs of interest, such as valves, myocardium, etc., can be accurately segmented, providing reliable basic data for subsequent three-dimensional reconstruction.

[0080] According to an embodiment of the present disclosure, for the n-th decoder, the n-th decoded feature is obtained by performing bilinear interpolation on the n-th encoded feature using the n-th decoder; where n ≥ 1.

[0081] As shown Figure 4As shown, in one example, four decoders are used to perform bilinear interpolation on the encoded features output by the corresponding four encoders respectively to obtain four decoded features. Bilinear interpolation is used to upsample all the encoded features to the original resolution. The first decoder performs bilinear interpolation on the encoded feature (B, 128, 112, 112) output by the first encoder to obtain the first decoded feature (B, C', 448, 448), where C' is the number of channels of the image and can be adjusted according to the application scenario; the second decoder performs bilinear interpolation on the encoded feature (B, 256, 56, 56) output by the second encoder to obtain the second decoded feature (B, C', 448, 448); the third decoder performs bilinear interpolation on the encoded feature (B, 512, 28, 28) output by the third encoder to obtain the third decoded feature (B, C', 448, 448); the fourth decoder performs bilinear interpolation on the encoded feature (B, 1024, 14, 14) output by the fourth encoder to obtain the fourth decoded feature (B, C', 448, 448). Bilinear interpolation can effectively generate smooth intermediate values by performing weighted averaging on adjacent pixels in the two-dimensional space, maintaining good image quality.

[0082] As Figure 4 shown, in one example, the four decoded features can be concatenated through concatenation fusion along the channel dimension to obtain a fused feature (B, 4C', 448, 448). The fused feature (B, 4C', 448, 448) is input into a convolutional network for convolution, batch normalization (Batch Normalization), and non-linear activation, and a semantic label map (B, 1, 448, 448) with the properties of a binary image is output.

[0083] According to an embodiment of the present disclosure, in the case where the target segmentation strategy is represented as a low-performance processing strategy, threshold segmentation processing is performed on the target ultrasound image to obtain a semantic label map.

[0084] According to an embodiment of the present disclosure, the segmentation processing can accurately separate the target region, providing precise basic data for subsequent three-dimensional reconstruction.

[0085] According to an embodiment of the present disclosure, in the case where the target image reconstruction strategy is represented as a low-performance processing strategy, surface rendering is performed on multiple semantic label maps to obtain a three-dimensional image model.

[0086] In one example, the operation of performing surface rendering on multiple semantic label maps is as follows:

[0087] Initialize the surface rendering mapper and use multiple semantic label maps as inputs to lay the foundation for subsequent isosurface extraction and rendering.

[0088] The isosurface is extracted using the Marching Cubes algorithm or other isosurface extraction algorithms from the image data. By setting a threshold, it is determined which pixels will be included in the isosurface, thereby generating the corresponding three-dimensional surface.

[0089] The color transfer function is initialized, and RGB points are added to define the color mapping. The color transfer function is used to map the grayscale values in the image data to specific colors, thereby displaying different tissues or structures in the three-dimensional model.

[0090] The opacity transfer function is initialized, and opacity points are added to define the transparency of different grayscale values, ensuring that parts with different grayscale values have appropriate transparency during the rendering process, making the three-dimensional model more hierarchical and detailed.

[0091] The face attributes are set. The face attributes are initialized, the color and opacity attributes are set, and the shadow effect is enabled. The boundary interpolation type is set to make the drawn face model smoother and ensure the rendering quality, which helps to improve the visual effect of the three-dimensional model, making it more realistic and delicate.

[0092] A renderer and a rendering window are created. The object is added to the renderer and displayed in the rendering window. The renderer is responsible for the actual drawing work, while the rendering window and the interactor provide the interface for the user to interact with the three-dimensional model.

[0093] Through the above process, the method of surface drawing is used to perform three-dimensional reconstruction of ultrasound. According to specific application requirements, parameters such as the settings of the color and opacity transfer functions and the threshold for isosurface extraction can be adjusted to obtain better visualization effects.

[0094] According to an embodiment of the present disclosure, in the case where the target image reconstruction strategy is represented as a low-performance processing strategy, volume rendering is performed on multiple semantic label maps to obtain a three-dimensional image model.

[0095] In one example, the operation of volume rendering for multiple semantic label maps using the isosurface volume rendering method is as follows:

[0096] The volume rendering mapper is initialized, and multiple semantic label maps are set as inputs.

[0097] The color and opacity are set. The color transfer function is initialized and RGB points are added to define the color mapping.

[0098] The piecewise function is initialized and opacity points are added to define the transparency of different grayscale values.

[0099] Set volume properties, initialize volume properties, set color and opacity properties, enable the shadow effect, and set the boundary interpolation type to make the rendered volume rendering model smoother and ensure the rendering quality.

[0100] Initialize the volume object and set the mapper and properties.

[0101] Initialize the renderer, render window, and interactor. Add the renderer to the render window and set the background color.

[0102] After completing the above settings, perform rendering and start the interaction operation to achieve three-dimensional reconstruction of ultrasonic images.

[0103] Figure 7 Schematically shows a chest model of three-dimensional reconstruction of an echocardiogram according to an embodiment of the present disclosure Figure 3 Schematic diagram of a three-dimensional reconstructed chest model of a three-dimensional reconstruction method of an echocardiogram.

[0104] As Figure 7 shown, using the three-dimensional reconstruction method of an echocardiogram according to an embodiment of the present disclosure Figure 3 reconstruct a three-dimensional chest model including ribs. Through rendering and interaction operations, three-dimensional reconstruction and interactive display of ultrasonic images are achieved, and files in STL format can be saved, facilitating subsequent 3D printing of organs or tissues. The three-dimensional reconstruction method of an echocardiogram according to an embodiment of the present disclosure Figure 3 has high image processing accuracy and rendering effect, and can be widely applied to medical imaging (such as CT or MRI, etc.) and other fields requiring three-dimensional reconstruction.

[0105] According to an embodiment of the present disclosure, an ultrasound workstation is configured with an ultrasound probe. When the ultrasound probe is set in the detection area of the object to be measured, in response to a detection instruction, the ultrasound probe is controlled to collect multiple frames of cardiac ultrasound images.

[0106] Figure 8 Schematically shows a block diagram of an echocardiogram Figure 3 three-dimensional reconstruction device according to an embodiment of the present disclosure.

[0107] As Figure 8 shown, the echocardiogram Figure 3 three-dimensional reconstruction device 800 includes a determination module 810, a preprocessing module 820, an image segmentation module 830, and a three-dimensional reconstruction module 840.

[0108] The determination module 810 is configured to respectively determine a target preprocessing strategy, a target segmentation strategy, and a target reconstruction strategy based on the remaining computing power resources of the ultrasound workstation.

[0109] The preprocessing module 820 is configured to preprocess multiple frames of cardiac ultrasound images of the object to be measured respectively based on the target preprocessing strategy to obtain multiple frames of target ultrasound images.

[0110] An image segmentation module 830, configured to perform image segmentation processing on a target ultrasound image based on a target segmentation strategy to obtain a semantic label map.

[0111] A three-dimensional reconstruction module 840, configured to perform three-dimensional reconstruction on multiple semantic label maps based on a target image reconstruction strategy to obtain a three-dimensional image model of the heart of the object under test.

[0112] According to an embodiment of the present disclosure, when the target segmentation strategy is represented as a high-performance processing strategy, the image segmentation module 830 includes a first obtaining sub-module, a second obtaining sub-module, a third obtaining sub-module, and a fourth obtaining sub-module.

[0113] The first obtaining sub-module is configured to process the target ultrasound image by using N encoders of an encoder-decoder network to obtain N encoded features, where N>1.

[0114] The second obtaining sub-module is configured to process the N encoded features by using N decoders of the encoder-decoder network respectively to obtain N decoded features.

[0115] The third obtaining sub-module is configured to splice the N decoded features to obtain a fused feature.

[0116] The fourth obtaining sub-module is configured to input the fused feature into a convolutional network to obtain a semantic label map.

[0117] According to an embodiment of the present disclosure, the first obtaining sub-module includes a first obtaining unit, a second obtaining unit, and a third obtaining unit.

[0118] The first obtaining unit is configured to, for the nth encoder, segment and linearly map the encoded feature output by the (n - 1)th encoder to obtain a feature sequence, where n≥1, and the encoded feature output by the 0th encoder is represented as the target ultrasound image.

[0119] The second obtaining unit is configured to process the feature sequence by using M network blocks of the nth encoder to obtain a target output sequence, where M≥1.

[0120] The third obtaining unit is configured to rearrange the target output sequence to obtain the encoded feature output by the nth encoder.

[0121] According to an embodiment of the present disclosure, the second obtaining unit includes a first obtaining sub-unit, a second obtaining sub-unit, a third obtaining sub-unit, a fourth obtaining sub-unit, a fifth obtaining sub-unit, a sixth obtaining sub-unit, and a seventh obtaining sub-unit.

[0122] The first obtaining subunit is configured to perform normalization processing on the input sequence of the m-th network block to obtain a normalized sequence, where m≥1, and the input sequence of the first network block is represented as a feature sequence.

[0123] The second obtaining subunit is configured to perform linear projection on the normalized sequence to obtain a first projection sequence and a second projection sequence.

[0124] The third obtaining subunit is configured to process the first projection sequence by using a forward convolutional layer and a forward state space layer to obtain a forward sequence.

[0125] The fourth obtaining subunit is configured to process the first projection sequence by using a backward convolutional layer and a backward state space layer to obtain a backward sequence.

[0126] The fifth obtaining subunit is configured to perform gating and merging processing on the forward sequence and the backward sequence based on the second projection sequence to obtain a first output sequence.

[0127] The sixth obtaining subunit is configured to perform downsampling processing, non-linear activation processing, and upsampling processing on the input sequence of the m-th network block in sequence to obtain a second output sequence.

[0128] The seventh obtaining subunit is configured to perform merging processing on the first output sequence and the second output sequence to obtain the output sequence of the m-th network block, where the output sequence of the M-th network block is represented as a target output sequence.

[0129] According to an embodiment of the present disclosure, the second obtaining sub-module includes a fourth obtaining unit.

[0130] The fourth obtaining unit is configured to, for the n-th decoder, perform bilinear interpolation on the n-th encoded feature by using the n-th decoder to obtain the n-th decoded feature; where n≥1.

[0131] According to an embodiment of the present disclosure, when the target segmentation strategy is represented as a low-performance processing strategy, the image segmentation module 830 includes a fifth obtaining sub-module.

[0132] The fifth obtaining sub-module is configured to perform threshold segmentation processing on the target ultrasound image to obtain a semantic label map.

[0133] According to an embodiment of the present disclosure, the 3D reconstruction module 840 includes a sixth obtaining sub-module and a seventh obtaining sub-module.

[0134] The sixth obtaining sub-module is configured to perform surface rendering on multiple semantic label maps to obtain a 3D image model when the target image reconstruction strategy is represented as a low-performance processing strategy.

[0135] A seventh obtaining sub-module, configured to perform volume rendering on multiple semantic label maps to obtain a three-dimensional image model when the target image reconstruction strategy is represented as a high-performance processing strategy.

[0136] According to an embodiment of the present disclosure, the preprocessing module 820 includes an eighth obtaining sub-module and a ninth obtaining sub-module.

[0137] The eighth obtaining sub-module is configured to perform adaptive histogram equalization processing on the cardiac ultrasound image to obtain a target ultrasound image when the target preprocessing strategy is represented as a low-performance processing strategy.

[0138] The ninth obtaining sub-module is configured to perform noise suppression on the cardiac ultrasound image by using a denoising network to obtain a target ultrasound image when the target preprocessing strategy is represented as a high-performance processing strategy.

[0139] According to an embodiment of the present disclosure, the ultrasound workstation is configured with an ultrasound probe, and the three-dimensional reconstruction device 800 of echocardiography Figure 3 further includes an acquisition module.

[0140] The acquisition module is configured to control the ultrasound probe to acquire multiple frames of cardiac ultrasound images in response to a detection instruction when the ultrasound probe is disposed in the detection area of the object to be measured.

[0141] According to the embodiments of the present disclosure, any multiple of the modules, sub-modules, units, and sub-units, or at least part of the functions of any of them can be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure can be split into multiple modules for implementation. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure can be at least partially implemented as a hardware circuit, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging circuits, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding functions can be executed.

[0142] For example, any combination of the determination module 810, the preprocessing module 820, the image segmentation module 830, and the three-dimensional reconstruction module 840 may be combined and implemented in one module / unit / sub-unit, or any one of the modules / units / sub-units may be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units may be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the determination module 810, the preprocessing module 820, the image segmentation module 830, and the three-dimensional reconstruction module 840 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable manner of integrating or packaging circuits, etc., implemented by hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in an appropriate combination of any several of them. Alternatively, at least one of the determination module 810, the preprocessing module 820, the image segmentation module 830, and the three-dimensional reconstruction module 840 may be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding functions may be executed.

[0143] It should be noted that the part of the three-dimensional reconstruction device in the embodiments of the present disclosure corresponds to the part of the three-dimensional reconstruction method in the embodiments of the present disclosure. For the description of the part of the three-dimensional reconstruction device, reference may be specifically made to the part of the three-dimensional reconstruction method, which will not be elaborated herein. Figure 3 The part of the three-dimensional reconstruction device in the embodiments of the present disclosure corresponds to the part of the three-dimensional reconstruction method in the embodiments of the present disclosure. Figure 3 For the description of the part of the three-dimensional reconstruction device, reference may be specifically made to the part of the three-dimensional reconstruction method. Figure 3 For the description of the part of the three-dimensional reconstruction device, reference may be specifically made to the part of the three-dimensional reconstruction method. Figure 3 which will not be elaborated herein.

[0144] Figure 9 Schematically shows a block diagram of an electronic device suitable for implementing the three-dimensional reconstruction method of echocardiography according to an embodiment of the present disclosure. Figure 3 The electronic device shown is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure. Figure 9 The electronic device shown is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0145] Such as Figure 9As shown, an electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage section 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include on-board memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of a method flow according to an embodiment of the present disclosure.

[0146] In the RAM 903, various programs and data required for the operation of the electronic device 900 are stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. The processor 901 performs various operations of a method flow according to an embodiment of the present disclosure by executing a program in the ROM 902 and / or the RAM 903. It should be noted that the program may also be stored in one or more memories other than the ROM 902 and the RAM 903. The processor 901 may also perform various operations of a method flow according to an embodiment of the present disclosure by executing a program stored in the one or more memories.

[0147] According to an embodiment of the present disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, and the input / output (I / O) interface 905 is also connected to the bus 904. The electronic device 900 may further include one or more of the following components connected to the input / output (I / O) interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed so that a computer program read from it can be installed into the storage section 908 as needed.

[0148] According to an embodiment of the present disclosure, the method flow according to the embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0149] The present disclosure also provides a computer-readable storage medium, which may be included in the device / device / system described in the above embodiment; or may exist separately without being assembled into the device / device / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0150] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or combined with an instruction execution system, device, or device.

[0151] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the above-described ROM 902 and / or RAM 903 and / or one or more memories other than ROM 902 and RAM 903.

[0152] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program includes program codes for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program codes are used to cause the electronic device to implement the method provided by the embodiment of the present disclosure.

[0153] When the computer program is executed by the processor 901, the above functions defined in the system / apparatus of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the above-described systems, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0154] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication section 909, and / or installed from the removable medium 911. The program code included in the computer program may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0155] According to embodiments of the present disclosure, the program code for executing the computer program provided by the embodiments of the present disclosure may be written in any combination of one or more programming languages. Specifically, these computing programs may be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include but are not limited to, such as Java, C++, Python, the "C" language, or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., by using an Internet service provider to connect through the Internet).

[0156] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions. Those skilled in the art can understand that the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0157] The embodiments of the present disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments have been described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present disclosure.

Claims

1. An echocardiogram three-dimensional reconstruction method, comprising: Based on the remaining computing power resources of the ultrasound workstation, respectively determining a target preprocessing strategy, a target segmentation strategy, and a target reconstruction strategy; For multiple frames of cardiac ultrasound images of the measured object obtained, based on the target preprocessing strategy, respectively preprocessing the multiple frames of cardiac ultrasound images to obtain multiple frames of target ultrasound images; Based on the target segmentation strategy, performing image segmentation processing on the target ultrasound images to obtain a semantic label map; and Based on the target image reconstruction strategy, performing three-dimensional reconstruction on multiple of the semantic label maps to obtain a three-dimensional image model of the heart of the measured object.

2. The method according to claim 1, wherein The performing image segmentation processing on the target ultrasound images based on the target segmentation strategy to obtain a semantic label map includes: When the target segmentation strategy is represented as a high-performance processing strategy, using N encoders of an encoder-decoder network to process the target ultrasound images to obtain N encoded features, where N>1; Using N decoders of the encoder-decoder network to process the N encoded features respectively to obtain N decoded features; Stitching the N decoded features to obtain a fused feature; and Inputting the fused feature into a convolutional network to obtain the semantic label map.

3. The method according to claim 2, wherein, The using N encoders of an encoder-decoder network to process the target ultrasound images to obtain N encoded features includes: For the nth encoder, segmenting and linearly mapping the encoded features output by the (n - 1)th encoder to obtain a feature sequence, where n≥1, and the encoded features output by the 0th encoder are represented as the target ultrasound images; Using M network blocks of the nth encoder to process the feature sequence to obtain a target output sequence, where M≥1; and Rearranging the target output sequence to obtain the encoded features output by the nth encoder.

4. The method according to claim 3, wherein The using M network blocks of the nth encoder to process the feature sequence to obtain a target output sequence includes: For the mth network block, performing normalization processing on the input sequence of the mth network block to obtain a normalized sequence, where m≥1, and the input sequence of the 1st network block is represented as the feature sequence; Performing linear projection on the normalized sequence to obtain a first projection sequence and a second projection sequence; Using a forward convolutional layer and a forward state space layer to process the first projection sequence to obtain a forward sequence; Using a backward convolutional layer and a backward state space layer to process the first projection sequence to obtain a backward sequence; Based on the second projection sequence, performing gating and merging processing on the forward sequence and the backward sequence to obtain a first output sequence; Performing downsampling processing, non-linear activation processing, and upsampling processing on the input sequence of the mth network block in sequence to obtain a second output sequence; and Performing merging processing on the first output sequence and the second output sequence to obtain the output sequence of the mth network block, where the output sequence of the Mth network block is represented as the target output sequence.

5. The method according to claim 2, wherein The N decoders of the encoder-decoder network process the N encoded features respectively to obtain N decoded features, including: For the nth decoder, bilinear interpolation is performed on the nth encoded feature by using the nth decoder to obtain the nth decoded feature; where n ≥ 1.

6. The method according to claim 1, wherein, Based on the target segmentation strategy, image segmentation processing is performed on the target ultrasound image to obtain a semantic label map, including: In the case where the target segmentation strategy is represented as a low-performance processing strategy, threshold segmentation processing is performed on the target ultrasound image to obtain the semantic label map.

7. The method according to claim 1, wherein, Based on the target image reconstruction strategy, three-dimensional reconstruction is performed on multiple semantic label maps to obtain a three-dimensional image model of the heart of the object under test, including: In the case where the target image reconstruction strategy is represented as a low-performance processing strategy, surface rendering is performed on multiple semantic label maps to obtain the three-dimensional image model; and In the case where the target image reconstruction strategy is represented as a high-performance processing strategy, volume rendering is performed on multiple semantic label maps to obtain the three-dimensional image model.

8. The method according to claim 1, wherein Based on the target preprocessing strategy, preprocessing is respectively performed on the multiple frames of cardiac ultrasound images to obtain multiple frames of target ultrasound images, including: In the case where the target preprocessing strategy is represented as a low-performance processing strategy, adaptive histogram equalization processing is performed on the cardiac ultrasound image to obtain the target ultrasound image; and In the case where the target preprocessing strategy is represented as a high-performance processing strategy, a denoising network is used to suppress noise in the cardiac ultrasound image to obtain the target ultrasound image.

9. The method according to claim 1, wherein, The ultrasound workstation is configured with an ultrasound probe, and the method further includes: In the case where the ultrasound probe is set in the detection area of the object under test, in response to a detection instruction, the ultrasound probe is controlled to collect the multiple frames of cardiac ultrasound images.

10. An echocardiogram three-dimensional reconstruction device, including: A determination module, configured to respectively determine a target preprocessing strategy, a target segmentation strategy, and a target reconstruction strategy based on the remaining computing power resources of the ultrasound workstation; A preprocessing module, configured to, for the multiple frames of cardiac ultrasound images of the object under test obtained, based on the target preprocessing strategy, respectively perform preprocessing on the multiple frames of cardiac ultrasound images to obtain multiple frames of target ultrasound images; An image segmentation module, configured to perform image segmentation processing on the target ultrasound image based on the target segmentation strategy to obtain a semantic label map; and A three-dimensional reconstruction module, configured to perform three-dimensional reconstruction on multiple semantic label maps based on the target image reconstruction strategy to obtain a three-dimensional image model of the heart of the object under test.

Images