Fetal heart region image segmentation method and system, electronic equipment and storage medium

Abstract

The invention provides a fetal heart region image segmentation method and system, electronic equipment and a storage medium, and the method comprises the steps: segmenting a current frame image of fetal heart body data through employing an inference network and a Canny edge detection algorithm; the fetal heart segmentation area of the current frame image is determined according to the obtained first fetal heart segmentation area and the second fetal heart segmentation area, and the inference network carries out inference based on the two-dimensional features and the three-dimensional features of the current frame image to obtain the first fetal heart segmentation area. The invention aims to combine two-dimensional information and three-dimensional information of an image so as to effectively solve the problems of non-fitting edges and poor reasoning effect of some frames in 3D segmentation, and a segmentation result of a Canny edge detection algorithm is used as a reference to help to judge and correct segmentation output generated by a reasoning network so as to improve the efficiency of 3D segmentation. By combining the advantages of the multi-source features and the Canny edge detection algorithm, the precision of complex heart structure segmentation is improved, and a more reliable segmentation result can be obtained.

Description

Technical Field

[0001] The present invention relates to the field of image segmentation, and more specifically, to a fetal heart region image segmentation method, system, electronic device and storage medium. Background Art

[0002] At present, the results of fetal heart segmentation have important scientific and application value in the clinical monitoring of fetal heart development. Fetal heart segmentation faces the challenges of two-dimensional and three-dimensional image processing. Traditional methods split three-dimensional data into multiple frames for processing, which can easily lead to loss of spatial information; the application of AI technology introduces a three-dimensional segmentation network that directly infers three-dimensional data, improving segmentation accuracy and consistency. In addition, combined with the 2D network processing strategy, three-dimensional image segmentation is achieved through post-reconstruction, which also shows high efficiency and flexibility.

[0003] In the process of fetal heart image segmentation, the single-frame 2D image segmentation method often fails to meet clinical needs due to the lack of deep support of the overall 3D network information. A significant limitation of this method is its inability to handle complex spatial relationships, which makes data distortion prone in subsequent 3D image reconstruction, which not only affects spatial consistency but also reduces the accuracy of diagnosis.

[0004] On the other hand, applying 3D network reasoning directly to 3D volume data, while being able to preserve the integrity of spatial information to some extent, also faces challenges. For example, the segmentation edges may not fit well enough, especially in the complex anatomical structure areas of the heart and the blurred edges caused by motion. In addition, due to the increase in data volume and computational complexity, the reasoning process may produce unsatisfactory results on some frames, which is reflected in the local inconsistency of segmentation accuracy. Summary of the invention

[0005] The present invention aims to solve the technical problems of edge mismatch and poor reasoning effect of some frames in 3D segmentation in the prior art, and provides a fetal heart region image segmentation method, system, electronic device and storage medium.

[0006] According to a first aspect of the present invention, a method for segmenting a fetal heart region image is provided, comprising: Splitting fetal heart volume data into different frame images; Using an inference network and a Canny edge detection algorithm for the current frame image, respectively, to obtain a first fetal heart segmentation region and a second fetal heart segmentation region of the current frame image, wherein the inference network infers the first fetal heart segmentation region based on two-dimensional features and three-dimensional features of the current frame image; Determine a fetal heart segmentation region of the current frame image according to the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image; The frame images of the fetal heart volume data are traversed frame by frame to obtain the fetal heart segmentation region of each frame image of the fetal heart volume data.

[0007] Based on the above technical solution, the present invention can also make the following improvements.

[0008] Optionally, the inference network includes an input layer, a two-dimensional U-Net network, a three-dimensional U-Net network, a fusion network and an output layer; The input layer is used to receive the input current frame image; The two-dimensional U-Net network is used to extract the two-dimensional features of the current frame image; The three-dimensional U-Net network is used to extract the three-dimensional features of the current frame image; The fusion network is used to fuse the two-dimensional features and the three-dimensional features to obtain a fusion feature map; The output layer is used to output a first fetal heart segmentation region according to the fused feature map.

[0009] Optionally, using a Canny edge detection algorithm on the current frame image to obtain a second fetal heart segmentation region of the current frame image includes: Detect edge information of the fetal heart region in the fetal heart volume data based on the Canny edge detection algorithm; The edge information of the detected fetal heart region is subjected to Gaussian smoothing to obtain a second fetal heart segmentation region.

[0010] Optionally, determining the fetal heart segmentation region of the current frame image according to the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image includes: If the first fetal heart segmentation area and the second fetal heart segmentation area are basically consistent, and the second fetal heart segmentation area is larger than the first fetal heart segmentation area, the second fetal heart segmentation area is used as the fetal heart segmentation area of ​​the current frame image; otherwise, the first fetal heart segmentation area is used as the fetal heart segmentation area of ​​the current frame image.

[0011] Optionally, if the first fetal heart segmentation region and the second fetal heart segmentation region are substantially consistent, and the second fetal heart segmentation region is larger than the first fetal heart segmentation region, using the second fetal heart segmentation region as the fetal heart segmentation region of the current frame image includes: Calculating a ratio A1 / A2 of the first fetal heart segmentation region A1 and the second fetal heart segmentation region A2 of the current frame image; If 0.95<A1 / A2<1, the second fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image; otherwise, the first fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image.

[0012] According to a second aspect of the present invention, there is provided a fetal heart region image segmentation system, comprising: A segmentation module, used for respectively using an inference network and a Canny edge detection algorithm for a current frame image of the fetal heart volume data to obtain a first fetal heart segmentation region and a second fetal heart segmentation region of the current frame image, wherein the inference network infers the first fetal heart segmentation region based on two-dimensional features and three-dimensional features of the current frame image; wherein the fetal heart volume data is split into different frame images; A determination module is used to determine the fetal heart segmentation area of ​​the current frame image according to the first fetal heart segmentation area and the second fetal heart segmentation area of ​​the current frame image; traverse the frame images of the fetal heart volume data frame by frame to obtain the fetal heart segmentation area of ​​each frame image of the fetal heart volume data.

[0013] According to a third aspect of the present invention, there is provided an electronic device comprising a memory and a processor, wherein the processor is configured to implement the steps of a fetal heart region image segmentation method when executing a computer management program stored in the memory.

[0014] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium on which a computer management program is stored, and when the computer management program is executed by a processor, the steps of the fetal heart region image segmentation method are implemented.

[0015] The present invention provides a method, system, electronic device and storage medium for fetal heart region image segmentation, which respectively uses an inference network and a Canny edge detection algorithm for the current frame image of the fetal heart volume data, and determines the fetal heart segmentation region of the current frame image based on the acquired first fetal heart segmentation region and the second fetal heart segmentation region, wherein the inference network infers the first fetal heart segmentation region based on the two-dimensional features and three-dimensional features of the current frame image. The present invention aims to combine the two-dimensional information and three-dimensional information of the image to effectively solve the problems of edge mismatch and poor reasoning effect of some frames in 3D segmentation, and uses the segmentation results of the Canny edge detection algorithm as a benchmark to help determine and correct the segmentation output generated by the inference network, and improves the accuracy of complex heart structure segmentation by combining the advantages of multi-source features and the Canny edge detection algorithm, which helps to obtain more reliable segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flow chart of a fetal heart region image segmentation method provided by the present invention; Figure 2 A schematic diagram of the overall architecture for fetal heart region image segmentation; Figure 3 It is a schematic diagram of the structure of the inference network; Figure 4 A schematic diagram of a block diagram of a fetal heart region image segmentation system provided by the present invention; Figure 5 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention; Figure 6 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be arbitrarily combined with each other to form a feasible technical solution. This combination is not subject to the constraints of the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in this field to achieve. When the combination of technical solutions is contradictory or cannot be achieved, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0018] The present invention designs an innovative segmentation network, which successfully integrates the two-dimensional (2D) information and three-dimensional (3D) information of fetal heart volume data, and obtains a more accurate segmentation result. Specifically, the method splits the fetal heart volume data into different frame data, and applies the traditional image segmentation algorithm to process each frame of the image. Subsequently, the segmentation results of the segmentation network and the traditional algorithm on each frame of the image are compared frame by frame. During the comparison process, if the segmentation results of the same frame image are basically the same, and the segmentation area of ​​the traditional algorithm is larger than the segmentation result inferred by the segmentation network, the segmentation result of the traditional algorithm is selected to replace the inference result of the segmentation network to ensure the integrity and reliability of the segmentation.

[0019] Figure 1 A flow chart of a fetal heart region image segmentation method provided by the present invention, such as Figure 1 and Figure 2 As shown, the method includes: Step 1: split the fetal heart volume data into different frame images.

[0020] Step 2: Use the inference network and the Canny edge detection algorithm for the current frame image respectively to obtain the first fetal heart segmentation area and the second fetal heart segmentation area of ​​the current frame image, wherein the inference network infers the first fetal heart segmentation area based on the two-dimensional features and three-dimensional features of the current frame image.

[0021] It is understandable that the collected fetal heart volume data is split into frames of image data.

[0022] For the current frame image, both the inference network and the traditional Canny edge detection algorithm are used for segmentation to obtain the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image respectively.

[0023] Among them, see Figure 3 , is an architecture diagram of an inference network. Exemplarily, the inference network includes an input layer, a two-dimensional U-Net network, a three-dimensional U-Net network, a fusion network and an output layer; An input layer is used to receive an input current frame image; a two-dimensional U-Net network is used to extract two-dimensional features of the current frame image; a three-dimensional U-Net network is used to extract three-dimensional features of the current frame image; a fusion network is used to fuse the two-dimensional features and the three-dimensional features to obtain a fused feature map; an output layer is used to output a first fetal heart segmentation area according to the fused feature map.

[0024] Specifically, the inference network enhances its performance by fusing the two-dimensional features and three-dimensional features of the current frame image. The extraction of two-dimensional features is performed by using a two-dimensional U-Net network, while the extraction of three-dimensional features is achieved by a three-dimensional U-Net network. In this inference network structure, different features are fused in a high-order feature map to obtain a fused feature map, and the segmentation result of the current frame image is obtained based on the fused feature map. This method not only improves the ability to capture spatial information, but also enhances the recognition and segmentation accuracy of target objects, especially in application fields such as medical image processing that require precise segmentation and detailed analysis.

[0025] Traditional image processing algorithm reasoning mainly relies on the Canny edge detection algorithm, which can effectively extract various boundary information in the image. After obtaining the edge information of different structures in the current frame image, the edge is smoothed by the Gaussian smoothing algorithm. Subsequently, the smoothed area is filled with different colors to finally obtain the reasoning result of the traditional algorithm. This method is widely used in image analysis and processing tasks, and provides a basic solution for various image processing needs.

[0026] Step 3: determine the fetal heart segmentation region of the current frame image according to the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image.

[0027] It is understandable that, for the first fetal heart segmentation region and the second fetal heart segmentation region obtained by using the inference network and the traditional Canny edge detection algorithm, the final fetal heart segmentation region is determined based on the comparison of the areas of the first fetal heart segmentation region and the second fetal heart segmentation region.

[0028] Exemplarily, if the first fetal heart segmentation region and the second fetal heart segmentation region are basically consistent, and the second fetal heart segmentation region is larger than the first fetal heart segmentation region, the second fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image; otherwise, the first fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image.

[0029] Wherein, a ratio A1 / A2 of the first fetal heart segmentation region A1 and the second fetal heart segmentation region A2 of the current frame image is calculated; If 0.95<A1 / A2<1, the second fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image; otherwise, the first fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image.

[0030] Specifically, by comparing the segmentation results of the inference network with those of the traditional algorithm, when the area ratio of the segmentation area of ​​the inference network to that of the traditional algorithm is greater than 0.95, and the area of ​​the segmentation area of ​​the traditional algorithm is larger than that of the inference network, the final segmentation result of the frame image will be based on the segmentation result of the traditional algorithm. For other cases, the final segmentation result of the frame image will be based on the segmentation result of the inference network. This strategy aims to optimize segmentation accuracy and ensure that a more accurate inference result is selected under the area judgment condition.

[0031] Step 4: traverse the frame images of the fetal heart volume data frame by frame to obtain the fetal heart segmentation region of each frame image of the fetal heart volume data.

[0032] It is understandable that the method of step 2 and step 3 is performed frame by frame for all frame images of the fetal heart volume data to obtain the fetal heart segmentation region of each frame image.

[0033] See also Figure 4 , provides a fetal heart region image segmentation system, including a segmentation module 401 and a determination module 402, wherein: The segmentation module 401 is used to use the inference network and the Canny edge detection algorithm for the current frame image of the fetal heart volume data to obtain the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image, wherein the inference network infers the first fetal heart segmentation region based on the two-dimensional features and the three-dimensional features of the current frame image; wherein the fetal heart volume data is split into different frame images; The determination module 402 is used to determine the fetal heart segmentation area of ​​the current frame image according to the first fetal heart segmentation area and the second fetal heart segmentation area of ​​the current frame image; traverse the frame images of the fetal heart volume data frame by frame to obtain the fetal heart segmentation area of ​​each frame image of the fetal heart volume data.

[0034] It can be understood that the fetal heart region image segmentation system provided by the present invention corresponds to the fetal heart region image segmentation method provided by the aforementioned embodiments. The relevant technical features of the fetal heart region image segmentation system can refer to the relevant technical features of the fetal heart region image segmentation method, which will not be repeated here.

[0035] See also Figure 5 , Figure 5 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 5 As shown, an embodiment of the present invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 511, the following steps are implemented: splitting the fetal heart volume data into different frame images; using an inference network and a Canny edge detection algorithm for the current frame image, respectively, to obtain a first fetal heart segmentation region and a second fetal heart segmentation region of the current frame image, wherein the inference network infers the first fetal heart segmentation region based on the two-dimensional features and the three-dimensional features of the current frame image; determining the fetal heart segmentation region of the current frame image according to the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image; traversing the frame images of the fetal heart volume data frame by frame to obtain the fetal heart segmentation region of each frame image of the fetal heart volume data.

[0036] See also Figure 6 , Figure 5 Schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. Figure 6As shown, this embodiment provides a computer-readable storage medium 600, on which a computer program 611 is stored. When the computer program 611 is executed by a processor, the following steps are implemented: splitting the fetal heart volume data into different frame images; using an inference network and a Canny edge detection algorithm for the current frame image respectively to obtain a first fetal heart segmentation region and a second fetal heart segmentation region of the current frame image, wherein the inference network infers the first fetal heart segmentation region based on the two-dimensional features and the three-dimensional features of the current frame image; determining the fetal heart segmentation region of the current frame image according to the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image; traversing the frame images of the fetal heart volume data frame by frame to obtain the fetal heart segmentation region of each frame image of the fetal heart volume data.

[0037] The fetal heart region image segmentation method, system, electronic device and storage medium provided by the embodiments of the present invention have the following beneficial effects: (1) Improving segmentation accuracy: By combining the 2D and 3D information of the image and utilizing multi-level feature fusion of deep learning, the accuracy of fetal heart image segmentation is significantly improved, especially at the edges of complex structures, achieving more detailed and accurate segmentation.

[0038] (2) Enhanced spatial consistency: Integrating multi-dimensional information can better maintain the consistency of three-dimensional space, thereby reducing data distortion during 3D reconstruction and improving the authenticity and reliability of the overall structure.

[0039] (3) Improved inter-frame reasoning: By integrating the advantages of deep learning and traditional algorithms, the problem of poor reasoning effect in individual frames is corrected to ensure consistent and accurate segmentation results across the entire frame.

[0040] (4) Enhanced clinical application value: Improved segmentation and reconstruction quality will directly promote the accuracy of clinical diagnosis and monitoring, enabling medical staff to screen and evaluate diseases earlier and more accurately, thereby improving pregnancy management and intervention strategies.

[0041] (5) Optimizing computational efficiency: With the coordinated optimization of multi-source information, the segmentation network can process large amounts of data more efficiently, shorten the computation and diagnosis time, and provide real-time or near real-time analysis capabilities.

[0042] (6) Providing innovative research paths: This technical solution provides a new research idea for other medical image segmentation tasks. Combining multi-faceted information and traditional methods, it is an important attempt to promote technological progress in the imaging field.

[0043] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0044] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0045] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0046] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0047] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0048] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0049] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A fetal heart region image segmentation method, characterized in that: include: Splitting fetal heart volume data into different frame images; Using an inference network and a Canny edge detection algorithm for the current frame image, respectively, to obtain a first fetal heart segmentation region and a second fetal heart segmentation region of the current frame image, wherein the inference network infers the first fetal heart segmentation region based on two-dimensional features and three-dimensional features of the current frame image; Determine a fetal heart segmentation region of the current frame image according to the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image; The frame images of the fetal heart volume data are traversed frame by frame to obtain the fetal heart segmentation region of each frame image of the fetal heart volume data.

2. The fetal heart region image segmentation method according to claim 1, characterized in that: The inference network includes an input layer, a two-dimensional U-Net network, a three-dimensional U-Net network, a fusion network and an output layer; The input layer is used to receive the input current frame image; The two-dimensional U-Net network is used to extract the two-dimensional features of the current frame image; The three-dimensional U-Net network is used to extract the three-dimensional features of the current frame image; The fusion network is used to fuse the two-dimensional features and the three-dimensional features to obtain a fusion feature map; The output layer is used to output a first fetal heart segmentation region according to the fused feature map.

3. The fetal heart region image segmentation method according to claim 1, characterized in that: The Canny edge detection algorithm is used for the current frame image to obtain the second fetal heart segmentation area of ​​the current frame image, including: Detect edge information of the fetal heart region in the fetal heart volume data based on the Canny edge detection algorithm; The edge information of the detected fetal heart region is subjected to Gaussian smoothing to obtain a second fetal heart segmentation region.

4. The fetal heart region image segmentation method according to claim 1, characterized in that: The step of determining the fetal heart segmentation region of the current frame image according to the first fetal heart segmentation region and the second fetal heart segmentation region of the current frame image comprises: If the first fetal heart segmentation area and the second fetal heart segmentation area are basically consistent, and the second fetal heart segmentation area is larger than the first fetal heart segmentation area, the second fetal heart segmentation area is used as the fetal heart segmentation area of ​​the current frame image; otherwise, the first fetal heart segmentation area is used as the fetal heart segmentation area of ​​the current frame image.

5. The fetal heart region image segmentation method according to claim 4, characterized in that: If the first fetal heart segmentation region and the second fetal heart segmentation region are substantially consistent, and the second fetal heart segmentation region is larger than the first fetal heart segmentation region, using the second fetal heart segmentation region as the fetal heart segmentation region of the current frame image, comprises: Calculating a ratio A1 / A2 of the first fetal heart segmentation region A1 and the second fetal heart segmentation region A2 of the current frame image; If 0.95<A1 / A2<1, the second fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image; otherwise, the first fetal heart segmentation region is used as the fetal heart segmentation region of the current frame image.

6. A fetal heart region image segmentation system, characterized in that: include: A segmentation module, used for respectively using an inference network and a Canny edge detection algorithm for a current frame image of the fetal heart volume data to obtain a first fetal heart segmentation region and a second fetal heart segmentation region of the current frame image, wherein the inference network infers the first fetal heart segmentation region based on two-dimensional features and three-dimensional features of the current frame image; wherein the fetal heart volume data is split into different frame images; A determination module is used to determine the fetal heart segmentation area of ​​the current frame image according to the first fetal heart segmentation area and the second fetal heart segmentation area of ​​the current frame image; traverse the frame images of the fetal heart volume data frame by frame to obtain the fetal heart segmentation area of ​​each frame image of the fetal heart volume data.

7. An electronic device, characterized in that: It comprises a memory and a processor, wherein the processor is used to implement the fetal heart region image segmentation method as claimed in any one of claims 1 to 5 when executing a computer management program stored in the memory.

8. A computer-readable storage medium, characterized in that: A computer management program is stored thereon, and when the computer management program is executed by the processor, the fetal heart region image segmentation method as described in any one of claims 1-5 is implemented.

Images