Fetal heart region image segmentation method, system, electronic device and storage medium

Abstract

The present application provides a kind of fetal heart region image segmentation method, system, electronic equipment and storage medium, respectively using inference network and Canny edge detection algorithm to the current frame image of fetal heart volume data is segmented, according to the first fetal heart segmentation area and the second fetal heart segmentation area obtained determine the fetal heart segmentation area of current frame image, inference network is based on the two-dimensional feature and three-dimensional feature of current frame image and inferences to obtain the first fetal heart segmentation area.The present application aims to combine the two-dimensional information and three-dimensional information of image, to effectively solve the problem of edge not fitting and some frames inference effect is poor in 3D segmentation, and the segmentation result of Canny edge detection algorithm is used as a benchmark, to help determine and correct the segmentation output generated by inference network, by the advantages of joint multi-source features and Canny edge detection algorithm, the precision of complex heart structure segmentation is improved, which helps to obtain more reliable segmentation result.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image segmentation, and more particularly, to a fetal heart region image segmentation method, system, electronic device and storage medium. BACKGROUND

[0002] At present, the fetal heart segmentation result has important scientific and application value in the clinical monitoring of the development process of the fetal heart. The fetal heart segmentation faces the challenges of two-dimensional and three-dimensional image processing. The traditional method disassembles the three-dimensional data into multiple frames for processing, which is easy to cause the loss of spatial information. The application of AI technology introduces a three-dimensional segmentation network that directly reasons on three-dimensional data, which improves the segmentation accuracy and consistency. In addition, combined with the strategy of 2D network processing, three-dimensional image segmentation is realized through post-reconstruction, which also shows high efficiency and flexibility.

[0003] In the fetal heart image segmentation process, the method of using single-frame 2D image for segmentation often cannot meet the clinical requirements due to the lack of deep support of overall 3D network information. A significant limitation of this method is its insufficient ability to handle complex spatial relationships, which can easily cause data distortion in subsequent 3D image reconstruction, affecting spatial consistency and reducing diagnostic accuracy.

[0004] On the other hand, directly applying 3D network reasoning to three-dimensional body data can preserve the integrity of spatial information to some extent, but also faces challenges. For example, the segmentation edge may not fit well, especially in the complex anatomical structure region of the heart and the edge blur zone caused by motion. In addition, due to the increase in data volume and computational complexity, the reasoning process may produce unsatisfactory results on some frames, which is manifested as local inconsistency in segmentation accuracy. SUMMARY

[0005] The present application provides a fetal heart region image segmentation method, system, electronic device and storage medium to solve the technical problems of edge mismatch in 3D segmentation and poor reasoning effect on some frames in the prior art.

[0006] According to a first aspect of the present application, a fetal heart region image segmentation method is provided, comprising:

[0007] splitting the fetal heart body data into different frame images;

[0008] using an inference network and a Canny edge detection algorithm on the current frame image respectively to obtain a first fetal heart segmentation region and a second fetal heart segmentation region of the current frame image, wherein the inference network is based on two-dimensional features and three-dimensional features of the current frame image to infer the first fetal heart segmentation region;

[0009] determine a fetal heart segmentation region of the current frame image according to the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image;

[0010] frame by frame, the fetal heart segmentation region of each frame image of the fetal heart volume data is obtained.

[0011] Based on the above technical solutions, the present application can also be improved as follows.

[0012] Optionally, the inference network comprises an input layer, a two-dimensional U-Net network, a three-dimensional U-Net network, a fusion network and an output layer.

[0013] The input layer is configured to receive an input current frame image.

[0014] The two-dimensional U-Net network is configured to extract two-dimensional features of the current frame image.

[0015] The three-dimensional U-Net network is configured to extract three-dimensional features of the current frame image.

[0016] The fusion network is configured to fuse the two-dimensional features and the three-dimensional features to obtain a fusion feature map.

[0017] The output layer is configured to output a first fetal heart segmentation region according to the fusion feature map.

[0018] Optionally, a Canny edge detection algorithm is used on the current frame image to obtain a second fetal heart segmentation region of the current frame image, comprising:

[0019] Edge information of a fetal heart region in the fetal heart volume data is detected based on the Canny edge detection algorithm.

[0020] The detected edge information of the fetal heart region is subjected to Gaussian smoothing processing to obtain the second fetal heart segmentation region.

[0021] Optionally, the determination of the fetal heart segmentation region of the current frame image according to the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image comprises:

[0022] If the first fetal heart segmentation region and the second fetal heart segmentation region are substantially consistent, and the second fetal heart segmentation region is larger than the first fetal heart segmentation region, the second fetal heart segmentation region is taken as the fetal heart segmentation region of the current frame image; otherwise, the first fetal heart segmentation region is taken as the fetal heart segmentation region of the current frame image.

[0023] Optionally, if the first fetal heart segmentation area and the second fetal heart segmentation area are substantially consistent, and the second fetal heart segmentation area is larger than the first fetal heart segmentation area, the second fetal heart segmentation area is taken as the fetal heart segmentation area of the current frame image, comprising:

[0024] The ratio A1 / A2 of the first fetal heart segmentation area A1 and the second fetal heart segmentation area A2 of the current frame image is calculated.

[0025] If 0.95

[0026] According to a second aspect of the present application, a fetal heart region image segmentation system is provided, comprising:

[0027] The segmentation module is configured to obtain a first fetal heart segmentation area and a second fetal heart segmentation area of a current frame image of the fetal heart volume data by using an inference network and a Canny edge detection algorithm respectively, wherein the inference network is based on two-dimensional features and three-dimensional features of the current frame image to infer the first fetal heart segmentation area; and the fetal heart volume data is split into different frame images.

[0028] The determination module is configured to determine a fetal heart segmentation area of the current frame image according to the first fetal heart segmentation area and the second fetal heart segmentation area of the current frame image, and to obtain the fetal heart segmentation area of each frame image of the fetal heart volume data by frame-by-frame traversing the frame images of the fetal heart volume data.

[0029] According to a third aspect of the present application, an electronic device is provided, comprising a memory and a processor, wherein the processor is configured to execute a computer management program stored in the memory to implement the steps of the fetal heart region image segmentation method.

[0030] According to a fourth aspect of the present application, a computer readable storage medium is provided, which stores a computer management program, wherein the computer management program is executed by a processor to implement the steps of the fetal heart region image segmentation method.

[0031] This invention provides a method, system, electronic device, and storage medium for fetal heart region image segmentation. It employs an inference network and the Canny edge detection algorithm on the current frame image of the fetal heart body data, and determines the fetal heart segmentation region of the current frame image based on the acquired first and second fetal heart segmentation regions. The inference network infers the first fetal heart segmentation region based on the two-dimensional and three-dimensional features of the current frame image. This invention aims to combine two-dimensional and three-dimensional information of the image to effectively solve the problems of edge misalignment and poor inference results in some frames during 3D segmentation. It also utilizes the segmentation results of the Canny edge detection algorithm as a benchmark to help determine and correct the segmentation output generated by the inference network. By combining the advantages of multi-source features and the Canny edge detection algorithm, the accuracy of segmenting complex heart structures is improved, contributing to more reliable segmentation results. Attached Figure Description

[0032] Figure 1 A flowchart of a fetal heart region image segmentation method provided by the present invention;

[0033] Figure 2 A schematic diagram of the overall architecture for segmenting images of the fetal heart region;

[0034] Figure 3 This is a schematic diagram of the inference network structure;

[0035] Figure 4 A schematic block diagram of a fetal heart region image segmentation system provided by the present invention;

[0036] Figure 5 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0037] Figure 6 This is a schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined with each other to form feasible technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0039] This invention designs an innovative segmentation network that successfully integrates two-dimensional (2D) and three-dimensional (3D) information from fetal heart body data, achieving relatively accurate segmentation results. Specifically, the method splits the fetal heart body data into different frames and processes each frame using a traditional image segmentation algorithm. Then, the segmentation results of the segmentation network and the traditional algorithm are compared frame by frame. During the comparison, if the segmentation results for the same frame are essentially consistent, and the segmented region of the traditional algorithm is larger than the segmentation result inferred by the segmentation network, the segmentation result of the traditional algorithm is selected instead of the inference result of the segmentation network to ensure the integrity and reliability of the segmentation.

[0040] Figure 1 A flowchart of a fetal heart region image segmentation method provided by the present invention is shown below. Figure 1 and Figure 2 As shown, the method includes:

[0041] Step 1: Split the fetal heart body data into different frame images.

[0042] Step 2: Apply an inference network and the Canny edge detection algorithm to the current frame image to obtain the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image. The inference network infers the first fetal heart segmentation region based on the two-dimensional and three-dimensional features of the current frame image.

[0043] Understandably, the collected fetal heart data is broken down into individual image frames.

[0044] For the current frame image, both the inference network and the traditional Canny edge detection algorithm are used for segmentation to obtain the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image, respectively.

[0045] Among them, see Figure 3 Here is an architecture diagram of an inference network. For example, the inference network includes an input layer, a two-dimensional U-Net network, a three-dimensional U-Net network, a fusion network, and an output layer.

[0046] The input layer receives the current frame image as input; the 2D U-Net network extracts 2D features from the current frame image; the 3D U-Net network extracts 3D features from the current frame image; the fusion network fuses the 2D and 3D features to obtain a fused feature map; and the output layer outputs the first fetal heart segmentation region based on the fused feature map.

[0047] Specifically, the inference network enhances its performance by fusing two-dimensional and three-dimensional features of the current frame image. Two-dimensional feature extraction is performed using a two-dimensional U-Net network, while three-dimensional feature extraction is achieved using a three-dimensional U-Net network. In this inference network structure, different features are fused in a higher-order feature map to obtain a fused feature map, and the segmentation result of the current frame image is obtained based on this fused feature map. This method not only improves the ability to capture spatial information but also enhances the accuracy of target object recognition and segmentation, especially in application areas requiring precise segmentation and detailed analysis, such as medical image processing.

[0048] Traditional image processing algorithms primarily rely on the Canny edge detection algorithm for inference, which effectively extracts various boundary information from images. After acquiring edge information of different structures in the current frame image, the edges are smoothed using a Gaussian smoothing algorithm. Subsequently, the smoothed regions are filled with different colors, ultimately obtaining the inference result of the traditional algorithm. This method is widely used in image analysis and processing tasks, providing a fundamental solution for various image processing needs.

[0049] Step 3: Determine the fetal heart segmentation region of the current frame image based on the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image.

[0050] Understandably, for the first and second fetal heart segmentation regions obtained using inference networks and the traditional Canny edge detection algorithm, the final fetal heart segmentation region is determined by comparing their areas.

[0051] For example, if the first fetal heart segmentation region and the second fetal heart segmentation region are substantially the same, and the second fetal heart segmentation region is larger than the first fetal heart segmentation region, then the second fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image; otherwise, the first fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image.

[0052] Among them, the ratio A1 / A2 of the first fetal heart segmentation region A1 and the second fetal heart segmentation region A2 of the current frame image is calculated;

[0053] If 0.95 < A1 / A2 < 1, then the second fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image; otherwise, the first fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image.

[0054] Specifically, when comparing the segmentation results of the inference network with those of the traditional algorithm, if the area ratio of the segmented region from the inference network to that from the traditional algorithm is greater than 0.95, and the area of ​​the segmented region from the traditional algorithm is larger than that from the inference network, the final segmentation result for that frame will be based on the traditional algorithm's result. In other cases, the final segmentation result for that frame will be based on the inference network's result. This strategy aims to optimize segmentation accuracy, ensuring that the more accurate inference result is selected under area-based determination conditions.

[0055] Step 4: Traverse the frame images of the fetal heart body data frame by frame to obtain the fetal heart segmentation region of each frame image of the fetal heart body data.

[0056] Understandably, steps 2 and 3 are performed frame by frame on all frames of fetal heart body data to obtain the segmented fetal heart region for each frame.

[0057] See Figure 4 A fetal heart region image segmentation system is provided, comprising a segmentation module 401 and a determination module 402, wherein:

[0058] The segmentation module 401 is used to obtain a first fetal heart segmentation region and a second fetal heart segmentation region of the current frame image of the fetal heart body data by using an inference network and the Canny edge detection algorithm respectively. The inference network infers the first fetal heart segmentation region based on the two-dimensional and three-dimensional features of the current frame image. The fetal heart body data is split into different frame images.

[0059] The determining module 402 is used to determine the fetal heart segmentation region of the current frame image based on the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image; and to traverse the frame images of the fetal heart body data frame by frame to obtain the fetal heart segmentation region of each frame image of the fetal heart body data.

[0060] It is understood that the fetal heart region image segmentation system provided by the present invention corresponds to the fetal heart region image segmentation method provided in the foregoing embodiments. The relevant technical features of the fetal heart region image segmentation system can be referred to the relevant technical features of the fetal heart region image segmentation method, and will not be repeated here.

[0061] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating an embodiment of the electronic device provided in this invention. For example... Figure 5 As shown, this embodiment of the invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 511, it performs the following steps: splitting fetal heart body data into different frame images; applying an inference network and a Canny edge detection algorithm to the current frame image to obtain a first fetal heart segmentation region and a second fetal heart segmentation region of the current frame image, wherein the inference network infers the first fetal heart segmentation region based on the two-dimensional and three-dimensional features of the current frame image; determining the fetal heart segmentation region of the current frame image based on the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image; and traversing the frame images of the fetal heart body data frame by frame to obtain the fetal heart segmentation region of each frame image of the fetal heart body data.

[0062] Please see Figure 6 , Figure 5 This is a schematic diagram illustrating an embodiment of a computer-readable storage medium provided by the present invention. (See diagram below.) Figure 6 As shown, this embodiment provides a computer-readable storage medium 600, on which a computer program 611 is stored. When the computer program 611 is executed by a processor, it performs the following steps: splitting fetal heart body data into different frame images; applying an inference network and a Canny edge detection algorithm to the current frame image to obtain a first fetal heart segmentation region and a second fetal heart segmentation region of the current frame image, wherein the inference network infers the first fetal heart segmentation region based on the two-dimensional and three-dimensional features of the current frame image; determining the fetal heart segmentation region of the current frame image based on the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image; and traversing the frame images of the fetal heart body data frame by frame to obtain the fetal heart segmentation region of each frame image of the fetal heart body data.

[0063] The fetal heart region image segmentation method, system, electronic device, and storage medium provided in this invention have the following beneficial effects:

[0064] (1) Improve segmentation accuracy: By combining the 2D and 3D information of the image and using the multi-level feature fusion of deep learning, the accuracy of fetal heart image segmentation is significantly improved, especially in the edge parts of complex structures, achieving more detailed and accurate segmentation.

[0065] (2) Enhance spatial consistency: Integrating multi-dimensional information can better maintain the consistency of three-dimensional space, thereby reducing data distortion and improving the authenticity and reliability of the overall structure during 3D reconstruction.

[0066] (3) Improve inter-frame inference performance: By combining the advantages of deep learning and traditional algorithms, the problem of poor inference performance in individual frames is corrected, ensuring consistent and accurate segmentation results across the entire frame.

[0067] (4) Enhance clinical application value: Improved segmentation and reconstruction quality will directly promote the accuracy of clinical diagnosis and monitoring, enabling medical staff to conduct disease screening and assessment earlier and more accurately, thereby improving pregnancy management and intervention strategies.

[0068] (5) Optimize computational efficiency: With the collaborative optimization of multi-source information, the segmentation network can process large amounts of data more efficiently, shorten the computation and diagnosis time, and provide real-time or near-real-time analysis capabilities.

[0069] (6) Providing innovative research paths: This technical solution provides a new research approach for other medical image segmentation tasks. Combining multi-dimensional information and traditional methods, it is an important attempt to promote the technological progress of the imaging field.

[0070] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0071] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0073] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0074] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0075] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0076] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for segmenting images of the fetal heart region, characterized in that, include: The fetal heart body data was split into different frame images; The inference network and the Canny edge detection algorithm are used to obtain the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image, respectively. The inference network infers the first fetal heart segmentation region based on the two-dimensional and three-dimensional features of the current frame image. Based on the first and second fetal heart segmentation regions of the current frame image, determine the fetal heart segmentation region of the current frame image; The frame images of the fetal heart body data are traversed frame by frame to obtain the fetal heart segmentation region of each frame image of the fetal heart body data. The inference network includes an input layer, a two-dimensional U-Net network, a three-dimensional U-Net network, a fusion network, and an output layer; The input layer is used to receive the current frame image as input; The two-dimensional U-Net network is used to extract two-dimensional features of the current frame image; The three-dimensional U-Net network is used to extract the three-dimensional features of the current frame image; The fusion network is used to fuse the two-dimensional features and the three-dimensional features to obtain a fused feature map; The output layer is used to output the first fetal heart segmentation region based on the fused feature map; The Canny edge detection algorithm is used on the current frame image to obtain the second fetal heart segmentation region of the current frame image, including: Edge information of the fetal heart region in fetal heart body data was detected based on the Canny edge detection algorithm; The edge information of the detected fetal heart region is smoothed using Gaussian to obtain the second segmented fetal heart region; Determining the fetal heart segmentation region of the current frame image based on the first and second fetal heart segmentation regions of the current frame image includes: If the first fetal heart segmentation region and the second fetal heart segmentation region are substantially the same, and the second fetal heart segmentation region is larger than the first fetal heart segmentation region, then the second fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image; otherwise, the first fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image.

2. The fetal heart region image segmentation method according to claim 1, characterized in that, If the first fetal heart segmentation region and the second fetal heart segmentation region are substantially the same, and the second fetal heart segmentation region is larger than the first fetal heart segmentation region, then the second fetal heart segmentation region is taken as the fetal heart segmentation region of the current frame image, including: Calculate the ratio A1 / A2 of the first fetal heart segmentation region A1 to the second fetal heart segmentation region A2 in the current frame image; If 0.95 < A1 / A2 < 1, then the second fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image; otherwise, the first fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image.

3. A fetal heart region image segmentation system, characterized in that, include: The segmentation module is used to obtain a first fetal heart segmentation region and a second fetal heart segmentation region from the current frame image of the fetal heart body data by using an inference network and the Canny edge detection algorithm respectively. The inference network infers the first fetal heart segmentation region based on the two-dimensional and three-dimensional features of the current frame image. The fetal heart body data is split into different frame images. The determination module is used to determine the fetal heart segmentation region of the current frame image based on the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image; and to traverse the frame images of the fetal heart body data frame by frame to obtain the fetal heart segmentation region of each frame image of the fetal heart body data. The inference network includes an input layer, a two-dimensional U-Net network, a three-dimensional U-Net network, a fusion network, and an output layer; The input layer is used to receive the current frame image as input; The two-dimensional U-Net network is used to extract two-dimensional features of the current frame image; The three-dimensional U-Net network is used to extract the three-dimensional features of the current frame image; The fusion network is used to fuse the two-dimensional features and the three-dimensional features to obtain a fused feature map; The output layer is used to output the first fetal heart segmentation region based on the fused feature map; The Canny edge detection algorithm is used on the current frame image to obtain the second fetal heart segmentation region of the current frame image, including: Edge information of the fetal heart region in fetal heart body data was detected based on the Canny edge detection algorithm; The edge information of the detected fetal heart region is smoothed using Gaussian to obtain the second segmented fetal heart region; Determining the fetal heart segmentation region of the current frame image based on the first and second fetal heart segmentation regions of the current frame image includes: If the first fetal heart segmentation region and the second fetal heart segmentation region are substantially the same, and the second fetal heart segmentation region is larger than the first fetal heart segmentation region, then the second fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image; otherwise, the first fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image.

4. An electronic device, characterized in that, It includes a memory and a processor, wherein the processor is used to implement the fetal heart region image segmentation method as described in claim 1 or 2 when executing a computer management program stored in the memory.

5. A computer-readable storage medium, characterized in that, It stores a computer management program, which, when executed by a processor, implements the fetal heart region image segmentation method as described in claim 1 or 2.

Images