A multi-branch segmentation system for cerebral hemorrhage and surrounding edema based on automatically generated labels

Abstract

The present invention relates to a multi-branch segmentation system for cerebral hemorrhage and surrounding edema based on automatically generated labels, which includes the following modules: a preprocessing module, which preprocesses the collected 3D cranial CT images; a label image generation module, which uses computer vision and machine learning algorithms to automatically generate cerebral hemorrhage label images and surrounding edema label images for the preprocessed 3D brain parenchyma CT images; a training module, which uses the automatically generated label images to train a multi-branch segmentation model for cerebral hemorrhage and surrounding edema based on deep learning, and then segments cerebral hemorrhage or surrounding edema or both at the same time; a measurement module, which uses the deep learning segmentation results to perform morphological measurements and present the results. The present invention combines computer vision and machine learning algorithms to automatically generate label images of cerebral hemorrhage and surrounding edema, and trains a multi-branch segmentation network based on the automatically labeled images, thereby obtaining accurate three-dimensional segmentation results of cerebral hemorrhage and surrounding edema.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a multi-branch segmentation system for cerebral hemorrhage and peripheral edema based on automatically generated labels. Background Art

[0002] Intracerebral hemorrhage (ICH) refers to primary, non-traumatic intraparenchymal bleeding, also known as spontaneous ICH. It is characterized by high morbidity, disability, and mortality. Accurate localization and volume measurement of intracranial hemorrhage (ICH) are paramount in the assessment of neurological dysfunction and brain injury. Perihematomal edema (PHE) is a key marker of secondary injury following ICH. Clinically, computed tomography (CT) has become the preferred modality for detecting ICH due to its low cost, short imaging time, excellent bone detail, and greater feasibility in unstable patients compared to magnetic resonance imaging (MRI). Currently, most hospitals primarily measure ICH using manual segmentation and the Tada formula. Manual calculation of ICH volume using the traditional Tada formula is cumbersome, inefficient, and poorly reproducible. Furthermore, this formula often overestimates the true ICH volume by up to 30%. Although manual segmentation of ICH can accurately estimate its volume, it is cumbersome and operator-dependent, and the operation time limits its feasibility in emergency situations.

[0003] Therefore, there is a need for an intelligent auxiliary cranial CT image segmentation and display system that can automatically complete the segmentation and measurement of cerebral hemorrhage and surrounding edema separately or simultaneously.

[0004] The above description of the background technology is only intended to facilitate an in-depth understanding of the technical solution of the present invention (such as the technical means used, the technical problems solved, and the technical effects produced), and should not be regarded as an admission or any form of implication that the information constitutes prior art already known to those skilled in the art. Summary of the Invention

[0005] In response to the defects in the existing technology, the present invention proposes an intelligent auxiliary brain CT image segmentation and display system that can automatically complete the segmentation and measurement of cerebral hemorrhage and surrounding edema separately or simultaneously. The system has strong universality and high intelligence, does not rely on manual or expert labels, and is not limited to CT brain data from a single source or a single scanning protocol.

[0006] According to an embodiment of the present invention, a multi-branch segmentation system for cerebral hemorrhage and surrounding edema based on automatically generated labels is provided, which includes the following modules: a preprocessing module, which preprocesses the collected 3D cranial CT images; a label image generation module, which uses computer vision and machine learning algorithms to automatically generate cerebral hemorrhage label images and surrounding edema label images for the preprocessed 3D brain parenchyma CT images; a training module, which uses the automatically generated cerebral hemorrhage label images and surrounding edema label images to train a multi-branch segmentation model for cerebral hemorrhage and surrounding edema based on deep learning, and then segments cerebral hemorrhage or surrounding edema or both at the same time; a measurement module, which uses the deep learning segmentation results of cerebral hemorrhage and surrounding edema to perform morphological measurements and present the results.

[0007] Preferably, in the preprocessing module, preprocessing of the acquired 3D brain CT images includes: data cleaning, format conversion, renaming, steering, zero voxel filling, registration, and window width and window position adjustment of the acquired 3D brain CT images, and finally extracting the brain parenchyma image.

[0008] Preferably, in the label image generation module, the threshold segmentation method and the cluster segmentation method are used respectively to generate the cerebral hemorrhage label image; generating the cerebral hemorrhage label image based on the threshold segmentation method is to smooth the preprocessed 3D brain parenchyma CT image, and then perform global threshold segmentation on the smoothed image; generating the cerebral hemorrhage label image based on the cluster segmentation method is to smooth the preprocessed 3D brain parenchyma CT image, and then perform fuzzy cluster segmentation on the smoothed image.

[0009] Preferably, in the label image generation module, the threshold segmentation method, the clustering segmentation method and the contralateral difference method are used to generate the peripheral edema label image respectively; the peripheral edema label image is generated based on the threshold segmentation method by expanding the cerebral hemorrhage label image of the same data outward according to a certain radius, and multiplying it with the preprocessed 3D brain parenchyma CT image to determine the initial edema area, and then performing single threshold segmentation on the initial edema area; the peripheral edema label image is generated based on the clustering segmentation method by expanding the cerebral hemorrhage label image of the same data outward according to a certain radius, and multiplying it with the preprocessed 3D brain parenchyma CT image to determine the initial edema area, and then performing fuzzy clustering segmentation on the initial edema area; the peripheral edema label image is generated based on the contralateral difference method by expanding the cerebral hemorrhage label image of the same data outward according to a certain radius, and multiplying it with the preprocessed 3D brain parenchyma CT image to determine the initial edema area, and then performing mirror reversal based on the midline to extract the corresponding area of ​​the contralateral brain, and finally subtracting the two areas, retaining the area with the difference within a certain range as the target edema area.

[0010] Preferably, the deep learning-based multi-branch segmentation model for cerebral hemorrhage and surrounding edema includes a segmentation network with an encoding-decoding U-shaped framework as the backbone network, wherein the segmentation network includes an encoding branch and multiple decoding branches with or without shared parameters, and the number of decoding branches is ≥2; the segmentation network is trained using automatically generated cerebral hemorrhage label images and surrounding edema label images, and an attention mechanism module or a Transformer module is provided at the output end of each decoding branch to integrate the preprocessed 3D brain parenchyma CT image information input to the segmentation network into the network.

[0011] Preferably, the deep learning-based multi-branch segmentation model for cerebral hemorrhage and surrounding edema further includes a positioning network, which uses an encoding-decoding U-shaped framework as the backbone network, and the positioning network includes an encoding branch and a decoding branch; the positioning network is trained using automatically generated cerebral hemorrhage label images and surrounding edema label images, and the image output by the decoding branch is dot-multiplied with the preprocessed 3D brain parenchyma CT image, and the resulting image is then input into the segmentation network to train the segmentation network.

[0012] Preferably, in the positioning network, the Dice similarity coefficient loss function is used; in the segmentation network, the average weighted or adaptively weighted Dice similarity coefficient loss function is used.

[0013] According to an embodiment of the present invention, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed by a processor, the following steps are implemented: preprocessing the acquired 3D cranial CT images; automatically generating cerebral hemorrhage label images and surrounding edema label images from the preprocessed 3D brain parenchyma CT images using computer vision and machine learning algorithms; using the automatically generated cerebral hemorrhage label images and surrounding edema label images to train a multi-branch segmentation model of cerebral hemorrhage and surrounding edema based on deep learning, and then segmenting cerebral hemorrhage or surrounding edema or both simultaneously; using the deep learning segmentation results of cerebral hemorrhage and surrounding edema, performing morphological measurements and presenting the results.

[0014] Preferably, preprocessing the acquired 3D brain CT image includes: performing data cleaning, format conversion, renaming, steering, zero voxel filling, registration, and window width and window position adjustment on the acquired 3D brain CT image, and finally extracting the brain parenchyma image.

[0015] Preferably, the threshold segmentation method and the clustering segmentation method are used respectively to generate a cerebral hemorrhage label image; generating a cerebral hemorrhage label image based on the threshold segmentation method is to smooth the preprocessed 3D brain parenchyma CT image, and then perform global threshold segmentation on the smoothed image; generating a cerebral hemorrhage label image based on the clustering segmentation method is to smooth the preprocessed 3D brain parenchyma CT image, and then perform fuzzy clustering segmentation on the smoothed image.

[0016] Preferably, the threshold segmentation method, the clustering segmentation method and the contralateral difference method are used to generate the peripheral edema label image respectively; the peripheral edema label image is generated based on the threshold segmentation method by expanding the cerebral hemorrhage label image of the same data outward according to a certain radius, and multiplying it with the preprocessed 3D brain parenchyma CT image to determine the initial edema area, and then performing single threshold segmentation on the initial edema area; the peripheral edema label image is generated based on the clustering segmentation method by expanding the cerebral hemorrhage label image of the same data outward according to a certain radius, and multiplying it with the preprocessed 3D brain parenchyma CT image to determine the initial edema area, and then performing fuzzy cluster segmentation on the initial edema area; the peripheral edema label image is generated based on the contralateral difference method by expanding the cerebral hemorrhage label image of the same data outward according to a certain radius, and multiplying it with the preprocessed 3D brain parenchyma CT image to determine the initial edema area, and then performing mirror reversal based on the midline to extract the corresponding area of ​​the contralateral brain, and finally subtracting the two areas, and retaining the area with the difference within a certain range as the target edema area.

[0017] Preferably, the deep learning-based multi-branch segmentation model for cerebral hemorrhage and surrounding edema includes a segmentation network with an encoding-decoding U-shaped framework as the backbone network, wherein the segmentation network includes an encoding branch and multiple decoding branches with or without shared parameters, and the number of decoding branches is ≥2; the segmentation network is trained using automatically generated cerebral hemorrhage label images and surrounding edema label images, and an attention mechanism module or a Transformer module is provided at the output end of each decoding branch to integrate the image information input to the segmentation network into the network.

[0018] Preferably, the deep learning-based multi-branch segmentation model for cerebral hemorrhage and surrounding edema further includes a positioning network, which uses an encoding-decoding U-shaped framework as the backbone network, and the positioning network includes an encoding branch and a decoding branch; the positioning network is trained using automatically generated cerebral hemorrhage label images and surrounding edema label images, and the image output by the decoding branch is dot-multiplied with the preprocessed 3D brain parenchyma CT image, and the resulting image is then input into the segmentation network to train the segmentation network.

[0019] Preferably, in the positioning network, the Dice similarity coefficient loss function is used; in the segmentation network, the average weighted and adaptive weighted Dice similarity coefficient loss function is used.

[0020] The present invention adopts the above technical solution, which has the following beneficial effects:

[0021] The present invention combines traditional machine learning algorithms to automatically generate labeled images of cerebral hemorrhage and surrounding edema, and trains a multi-branch deep learning network based on the automatically labeled images. It can obtain accurate three-dimensional segmentation results of cerebral hemorrhage and surrounding edema, perform morphological measurements based on the segmentation results, and perform three-dimensional skull, cerebral hemorrhage, and surrounding edema superposition (perspective) display. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The following will describe exemplary embodiments of the present invention in more detail with reference to the accompanying drawings. For clarity, identical components in different drawings are represented by the same reference numerals. It should be noted that the drawings are for illustrative purposes only and are not necessarily drawn to scale. In these drawings:

[0023] Figure 1 is a schematic diagram of the working principle of a multi-branch segmentation system for cerebral hemorrhage and surrounding edema based on automatically generated labels according to an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of a process of preprocessing raw data by a multi-branch segmentation system for cerebral hemorrhage and peripheral edema based on automatically generated labels according to an embodiment of the present invention;

[0025] Figure 3(a) to Figure 3(h) is a schematic diagram of preprocessing results of a multi-branch segmentation system for cerebral hemorrhage and surrounding areas based on automatically generated labels, taking two data sets as examples according to an embodiment of the present invention;

[0026] Figure 4 is a schematic diagram of a multi-branch segmentation model for cerebral hemorrhage and surrounding edema based on deep learning according to an embodiment of the present invention;

[0027] Figure 5(a) to Figure 5(f) and Figure 6(a) to Figure 6(f) 3 is a schematic diagram of segmentation results of a multi-branch segmentation system for cerebral hemorrhage and peripheral edema based on automatically generated labels according to an embodiment of the present invention, taking two data sets as examples. DETAILED DESCRIPTION

[0028] The following is a detailed description of the implementation scheme of the present invention. This implementation scheme is implemented on the premise of the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following implementation scheme.

[0029] Figure 1 Schematic diagram of the working principle of a multi-branch segmentation system for cerebral hemorrhage and surrounding edema based on automatically generated labels according to an embodiment of the present invention. The working principle of the multi-branch segmentation system for cerebral hemorrhage and surrounding edema based on automatically generated labels according to the present invention is as follows and is mainly divided into four modules.

[0030] A preprocessing module, which collects 3D cranial CT images and preprocesses the collected 3D cranial CT images;

[0031] A label image generation module, which uses computer vision and machine learning algorithms to automatically generate cerebral hemorrhage label images and surrounding edema label images from preprocessed 3D brain parenchyma CT images;

[0032] A training module that uses automatically generated labeled images to train a deep learning network combined with an attention mechanism or a Transformer module to perform (semantic) segmentation of intracerebral hemorrhage, surrounding edema, or both;

[0033] The measurement module uses the deep learning segmentation results of cerebral hemorrhage and surrounding edema to perform morphological measurements (e.g., volume, surface area, average width, thickness, specific surface area, etc.) and present the results.

[0034] The processing of each module of the multi-branch segmentation system for cerebral hemorrhage and peripheral edema based on automatically generated labels of the present invention is described in detail below.

[0035] Figure 2 The figure shows the process of preprocessing the original 3D brain CT image by the multi-branch segmentation system of cerebral hemorrhage and surrounding edema based on automatically generated labels according to the present invention.

[0036] like Figure 2 As shown, the multi-branch segmentation system for cerebral hemorrhage and surrounding edema based on automatically generated labels according to the present invention can be applied to 3D cranial CT images, and its data format can include but is not limited to DICOM (Digital Imaging and Communications in Medicine) format and NIFTI (Neuroimaging Informatics Technology Initiative) format, and does not distinguish between scanning settings such as image resolution, layer thickness, peak voltage (Kilovolt Peak, KVP) (120, 130, 140KVP), that is, there is no restriction on the scanning protocol.

[0037] Preprocessing of acquired 3D brain CT images involves data cleaning, format conversion, renaming, image rotation, zero voxel filling, registration, and window width and level adjustment, ultimately extracting the brain parenchyma image. Rotation and window width and level adjustment are optional and can be selected as needed.

[0038] Data cleaning involves excluding data that has undergone early treatment (e.g., drainage) or has severe quality issues (e.g., artifacts and ghosting). The remaining data can be used. All usable data is then subjected to the following steps: format conversion, renaming, rotation, zero voxel filling, registration, and window width and level adjustment.

[0039] According to an embodiment of the present invention, if the available data is in DICOM format, it is converted into NIFTI format data; if the available data is in NIFTI format, no format conversion processing is performed. The image size is unified by zero voxel filling processing, and the image size with a layer thickness less than 4 mm is uniformly changed to 512×512×150 voxels; the image size with a layer thickness greater than or equal to 4 mm is uniformly changed to 512×512×40 voxels. The registration of medical images is the process of converting different images of the same scene into the same coordinate system. This article uses the pre-processed selected image (i.e., the image in which the brain midline in the grayscale image coincides with the image geometric midline or the cross-sectional midline) as a reference, and the remaining images are used as moving images to complete the registration process so that the brain anatomical structure of each data corresponds to each other in the same coordinate space.

[0040] Figure 3(a) to Figure 3(h) Figure 3(a) and Figure 3(e) show two original 3D brain CT images, Figure 3(b) and Figure 3(f) show the images obtained after registration, Figure 3(c) and Figure 3(g) show the images obtained after adjusting the window width and window position, and Figure 3(d) and Figure 3(h) show the final brain parenchyma images.

[0041] Hereinafter, the automatic generation of a label image using a pre-processed 3D brain parenchymal CT image will be described.

[0042] After preprocessing the original 3D cranial CT image (i.e., obtaining the brain parenchyma image), computer vision and machine learning algorithms are used to automatically generate cerebral hemorrhage label images and peripheral hematoma label images from the preprocessed 3D brain parenchyma CT image.

[0043] According to the image characteristics of cerebral hemorrhage, since the high-density area of ​​cerebral hemorrhage (i.e., blood coagulation) can contrast with the adjacent tissue, methods such as threshold segmentation method and cluster segmentation method can be used to locate and segment the boundaries of cerebral hemorrhage. The following uses the threshold segmentation method and cluster segmentation method as examples to describe the location and boundary segmentation of cerebral hemorrhage.

[0044] The threshold segmentation method for generating a brain hemorrhage label image uses a single threshold segmentation method based on the CT value range of general brain hemorrhage, thereby dividing the CT image into two major categories: brain hemorrhage and background (here, the background includes the brain parenchyma and other non-brain areas). In an embodiment of the present invention, the pre-processed 3D brain parenchyma CT image is first smoothed. Here, Gaussian smoothing is used, and the maximum range of the smoothing coefficient (sigma) is (0, 5), preferably [1, 4], and in this embodiment, 3.5. The smoothed image is then subjected to global threshold segmentation. Here, the threshold is fixed in the range of [40, 100] CT values.

[0045] The generation of cerebral hemorrhage label images based on the cluster segmentation method is to generate multiple classifications based on the pre-processed 3D brain parenchyma CT image, and continuously correct the cluster center by repeatedly calculating the membership of each voxel point in the image, so that the objective function reaches the optimal solution. Among them, the last category is cerebral hemorrhage. In the embodiment of the present invention, the pre-processed 3D brain parenchyma CT image is first smoothed, and Gaussian smoothing is used here. The maximum range value of the smoothing coefficient (sigma) is (0, 5], preferably [1, 3], and is selected as 1.5 in this embodiment; the smoothed image is fuzzy clustered and segmented, and the maximum range of the number of clusters is [3, 10], preferably [3, 8], and is selected as 5 in this embodiment.

[0046] The embodiment of the present invention utilizes a threshold segmentation method and a cluster segmentation method that are closely related to the difference in 3D brain CT values ​​to locate and segment the boundaries of cerebral hemorrhage, but the present invention is not limited to the above threshold segmentation method and cluster segmentation method.

[0047] Using the above-mentioned computer vision and machine learning methods, such as the threshold segmentation method and the clustering segmentation method, the brain hemorrhage image can be segmented. Unlike the characteristics of the brain hemorrhage image, the peripheral edema image has a large overlap with the adjacent cerebrospinal fluid and white matter grayscale images, resulting in unclear boundaries of the peripheral edema image. Therefore, it is difficult to segment the peripheral edema area. The embodiment of the present invention converts the principles and experience of judging peripheral edema into the edema segmentation algorithm (the edema segmentation algorithms mentioned here include but are not limited to the three algorithms of threshold segmentation, clustering, and contralateral difference), thereby increasing the interpretability and reliability of the results.

[0048] The following describes the positioning and boundary segmentation of peripheral edema using the threshold segmentation method, clustering segmentation method and contralateral difference method as examples. The embodiment of the present invention performs positioning and boundary segmentation of peripheral edema based on the cerebral hemorrhage image of the same data.

[0049] The threshold segmentation method generates a label image of peripheral edema by expanding the cerebral hemorrhage data outward according to a certain radius and performing a dot product with the preprocessed 3D brain parenchyma CT image to determine the initial edema region. Then, using a threshold range for edema surrounding a typical blood clot, a single threshold segmentation is performed on the initial edema region, dividing it into two categories: edema and background (here, background includes brain parenchyma and other non-brain areas). In the present invention, the maximum range of the radius of the outward expansion of the cerebral hemorrhage is (0, 50) voxels, preferably (10, 30) voxels, and in this embodiment, 20 voxels. The initial edema region after the dot product is then threshold segmented, where the threshold is fixed within the range of [5, 33] CT values.

[0050] The method of generating a peripheral edema label image based on the cluster segmentation method is to expand the cerebral hemorrhage of the same data outward according to a certain radius, and determine the initial area of ​​edema by dot multiplication with the pre-processed 3D brain parenchyma CT image, and then generate multiple classifications based on the initial area of ​​edema. By repeatedly calculating the membership of each voxel point in the image, the cluster center is continuously corrected, so that the objective function reaches the optimal solution. Among them, the last category is peripheral edema. In the present invention, the maximum range value of the radius of the outward expansion of cerebral hemorrhage is (0, 50] voxels, preferably [10, 30] voxels, and in this embodiment, 20 voxels are selected; the initial area of ​​edema is fuzzy clustered and segmented, and the maximum range of the number of clusters is [3, 10], preferably [6, 10], and in this embodiment, 8 is selected.

[0051] The contralateral difference method generates a peripheral edema label image by expanding the cerebral hemorrhage on the same 3D cranial CT image outward according to a certain radius and multiplying it with the preprocessed 3D brain parenchyma CT image to determine the initial edema area. A mirror image is then inverted based on the midline to extract the corresponding area on the contralateral side of the brain. Finally, the two areas are subtracted, and the area with the difference within the specified range is retained as the target edema area. In an embodiment of the present invention, the maximum range of the radius of the outward expansion of the cerebral hemorrhage is (0, 50) voxels, with a preferred range of [10, 30] voxels, and in this embodiment, 20 voxels are selected; all voxels with a difference greater than 0 between the two areas are retained.

[0052] The implementation scheme of the present invention utilizes the threshold segmentation method, cluster segmentation method and contralateral difference method to locate and segment the boundaries of peripheral edema according to the image characteristics of peripheral edema, but the present invention is not limited to the above-mentioned threshold segmentation method, cluster segmentation method and contralateral difference method.

[0053] In the generation of labeled images for cerebral hemorrhage and surrounding edema described above, computer vision and machine learning algorithms can be used to quickly generate automatic labels, eliminating the need for manual annotation. In the following, we will use the automatically generated labeled images to train a deep learning network combined with an attention mechanism or Transformer module to perform (semantic) segmentation of cerebral hemorrhage, surrounding edema, or both.

[0054] That is, if only the cerebral hemorrhage label image is used, the cerebral hemorrhage area can be segmented using the deep learning network; if only the surrounding edema label image is used, the surrounding edema image can be segmented using the deep learning network; if the composite label image of the cerebral hemorrhage label image and the surrounding edema image is used (which contains both the cerebral hemorrhage label image and the surrounding edema label image), the surrounding edema image can be segmented at the same time using the deep learning network.

[0055] Figure 4 A schematic diagram of a multi-branch segmentation model for cerebral hemorrhage and surrounding edema based on deep learning is shown. The multi-branch segmentation model for cerebral hemorrhage and surrounding edema based on deep learning according to an embodiment of the present invention combines an attention mechanism or a Transformer module with a U-shaped deep learning network structure, and can segment cerebral hemorrhage and surrounding edema separately or simultaneously.

[0056] According to an embodiment of the present invention, a multi-branch segmentation model for cerebral hemorrhage and peripheral edema based on deep learning uses an encoding-decoding U-shaped framework as the backbone network, wherein the encoder and decoder can be any fully convolutional neural network, such as Residual Neural Network (ResNet), Fully Convolutional Networks (FCN), U-Net, etc.

[0057] The multi-branch segmentation model for cerebral hemorrhage and peripheral edema based on deep learning can include a localization network and a segmentation network, wherein the segmentation network can be used independently to complete the same task (i.e., the multi-branch segmentation model can only include the segmentation network).

[0058] The localization network uses a U-shaped encoding-decoding framework as its backbone, consisting of an encoding branch and a decoding branch. Automatically generated labeled images of ICH and surrounding edema are used to train the localization network. The decoder branch output image (a binary image containing ICH and surrounding edema segmented by the localization network, i.e., the localization box) is then dot-multiplied with a preprocessed 3D brain parenchymal CT image. The resulting image is then fed into the segmentation network to further train the network and reduce background areas outside the region of interest (i.e., the ICH and surrounding edema).

[0059] In the positioning network, the Dice coefficient loss function is used.

[0060] The segmentation network is designed as a multi-branch network structure according to the number (≥2) of non-manual labels (automatic labels). The network includes an encoding branch and a plurality of decoding branches with shared or unshared parameters, thereby integrating the information of multiple automatic labels to enhance the fitting ability of the network architecture. According to an embodiment of the present invention, the number of decoding branches is ≥2, preferably 2 to 10. In addition, an attention mechanism module or a Transformer module is provided at the output end of the decoding branch to integrate the information of the pre-processed 3D brain parenchyma CT image into the network, and perform weighted changes on the target data to improve the accuracy of network segmentation. According to an embodiment of the present invention, the attention mechanism module or the Transformer module can be, but is not limited to, an attention gate module.

[0061] By assigning different loss factors to each branch, the network can assign different weights based on the importance of the labels of each branch. In the embodiment of the present invention, for the multi-branch structure, average weighted and adaptive weighted loss strategies are introduced in the segmentation network to ensure that the model can learn the input training samples more quickly and effectively.

[0062] The average / constant weighted loss value is calculated as follows:

[0063]

[0064] Where n=2,3,4…, n represents the number of non-manual labels. i Represents the Dice similarity coefficient loss function value corresponding to each label image branch.

[0065] The adaptive weighted loss value is calculated as follows:

[0066]

[0067]

[0068] In an embodiment of the present invention, the weight value may be set according to the Dice similarity coefficient loss function value of each label image, that is, the higher the Dice similarity coefficient loss function value of the label image, the greater its weight value.

[0069] The initial convolution kernel weights are obtained from a Gaussian distribution, and then the model is optimized using a stochastic gradient descent optimizer, and the Dice similarity coefficient is applied to the final feature map for the calculation of the loss function.

[0070] The multi-branch deep learning network structure according to the embodiment of the present invention is based on a U-shaped network framework of encoding and decoding. It includes an attention mechanism or Transformer architecture of an encoding branch and multiple decoding branches for multiple non-manual cerebral hemorrhage and surrounding edema labels. The network and each branch pay more attention to the accuracy of the segmentation results of cerebral hemorrhage and surrounding edema. In the non-manual label generation part, many different non-manual labels for the same brain CT image are obtained. During the training process, the multi-branch deep learning network learns multiple non-manual labels at the same time, so that the network can generate a new and more accurate segmentation result from these complementary labels, and improve the overall accuracy by introducing an attention mechanism or Transformer architecture and an adaptive weighted loss function.

[0071] Figure 5(a) to Figure 5(f) and Figure 6(a) to Figure 6(f) 3 is a schematic diagram of segmentation results of a multi-branch segmentation system for cerebral hemorrhage and peripheral edema based on automatically generated labels according to an embodiment of the present invention, taking two data sets as examples.

[0072] According to an embodiment of the present invention, a multi-branch segmentation system for cerebral hemorrhage and surrounding edema based on deep learning can be used to segment the image and obtain the segmentation results of cerebral hemorrhage and surrounding edema (wherein the background value is 0, the hemorrhage value is 1, and the edema value is 2). Thus, a three-dimensional skull, cerebral hemorrhage, and surrounding edema are superimposed (perspective) displayed in combination with the collected 3D cranial CT image. Wherein, Figures 5(a) and 6(a) represent two-dimensional collected 3D cranial CT images; Figures 5(b) and 6(b) represent two-dimensional superimposed display images of cerebral hemorrhage and surrounding edema with cerebral hemorrhage CT images; Figures 5(c) and 6(c) represent three-dimensional perspective views of cerebral hemorrhage; Figures 5(e) and 6(e) represent three-dimensional perspective views of surrounding edema; Figures 5(d) and 6(d) represent three-dimensional perspective views of cerebral hemorrhage and surrounding edema superimposed; Figures 5(f) and 6(f) represent three-dimensional perspective views of cerebral hemorrhage and surrounding edema superimposed with the skull.

[0073] After the segmentation results are obtained, morphological measurements can also be performed, such as volume, surface area, average width, thickness, specific surface area, etc. The volume is calculated by multiplying the number of voxels with a label value of 1 by the volume of a single voxel. The surface area is calculated by reconstructing the segmentation results using the Marching cube surface rendering algorithm to obtain a three-dimensional model surface composed of several triangular facets. The area of ​​each triangular facet can be calculated using Heron's formula. The total surface area can be obtained by summing the areas of all triangular facets. The remaining indicators are calculated based on volume and surface area.

[0074] Therefore, the multi-branch segmentation system of cerebral hemorrhage and surrounding edema based on automatically generated labels of the present invention can automatically generate label images of cerebral hemorrhage and surrounding edema in combination with the algorithm of computer vision and machine learning, and train the multi-branch deep learning network based on the automatic label image to obtain accurate cerebral hemorrhage and surrounding edema 3D segmentation results, and can perform three-dimensional skull, cerebral hemorrhage, surrounding edema superposition (perspective) display. The system has strong universality and high intelligent degree, does not rely on manual or expert labels, and is not limited to CT brain data from a single source or a single scanning protocol.

[0075] Although the exemplary method of the present invention described above is represented as a series of operations for the sake of clarity, it is not intended to limit the order in which the steps are performed, and each step can be performed simultaneously or in a different order as desired. In order to implement the method according to the present invention, the steps shown may further include other steps, may include the remaining steps other than certain steps, or may include other additional steps other than certain steps.

[0076] The various embodiments of the invention are not an exhaustive list of all possible combinations, but are intended to describe representative aspects of the invention, and what is described in terms of various embodiments can be applied independently or in combinations of two or more.

[0077] In addition, various embodiments of the present invention may be implemented by hardware, firmware, software, or a combination thereof. The hardware may be implemented by one or more of a graphics processing unit (GPU) or other application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a general purpose processor, a controller, a microcontroller, a microprocessor, and the like.

[0078] The scope of the present invention is intended to include software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that enable operations according to various embodiments to be performed on a device or computer, as well as non-transitory computer-readable media executable on a device or computer having such software or instructions, etc. stored thereon.

[0079] The descriptions presented in the above exemplary embodiments are only intended to illustrate the technical solutions of the present invention and are not intended to be exhaustive or to limit the present invention to the precise forms described. Obviously, it is possible for a person of ordinary skill in the art to make many changes and variations based on the above teachings. The exemplary embodiments are selected and described to explain the specific principles of the present invention and its practical applications, so that other persons of ordinary skill in the art can easily understand, implement and utilize the various exemplary embodiments of the present invention and its various selected forms and modified forms. The scope of protection of the present invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A multi-branch segmentation system for cerebral hemorrhage and surrounding edema based on automatically generated labels, characterized by Includes the following modules: A preprocessing module, which preprocesses the collected 3D brain CT images; A label image generation module, which uses computer vision and machine learning algorithms to automatically generate cerebral hemorrhage label images and surrounding edema label images from preprocessed 3D brain parenchyma CT images; A training module uses automatically generated labeled images of cerebral hemorrhage and surrounding edema to train a deep learning-based multi-branch segmentation model for cerebral hemorrhage and surrounding edema, thereby simultaneously segmenting both cerebral hemorrhage and surrounding edema. The measurement module uses the deep learning segmentation results of cerebral hemorrhage and surrounding edema to perform morphological measurements and present the results. The deep learning-based multi-branch segmentation model for cerebral hemorrhage and peripheral edema includes a segmentation network with an encoding-decoding U-shaped framework as the backbone network. The segmentation network includes an encoding branch and multiple decoding branches with or without shared parameters, and the number of decoding branches is ≥2. The segmentation network is trained using automatically generated labeled images of cerebral hemorrhage and surrounding edema. An attention mechanism module or a Transformer module is set at the output of each decoding branch to integrate the information of the preprocessed 3D brain parenchyma CT image into the segmentation network. The deep learning-based multi-branch segmentation model for cerebral hemorrhage and peripheral edema further includes a localization network with an encoding-decoding U-shaped framework as a backbone network, wherein the localization network includes an encoding branch and a decoding branch; The automatically generated cerebral hemorrhage label image and surrounding edema label image are used to train the localization network, and the image output by the decoding branch is multiplied with the preprocessed 3D brain parenchyma CT image, and the resulting image is then input into the segmentation network to train the segmentation network.

2. The multi-branch segmentation system for cerebral hemorrhage and peripheral edema based on automatically generated labels according to claim 1, characterized in that: In the preprocessing module, The preprocessing of the acquired 3D brain CT images includes: data cleaning, format conversion, renaming, steering, zero voxel filling, registration, and window width and window position adjustment of the acquired 3D brain CT images, and finally extracting the brain parenchyma image.

3. The multi-branch segmentation system for cerebral hemorrhage and peripheral edema based on automatically generated labels according to claim 1, characterized in that: In the label image generation module, the threshold segmentation method and clustering segmentation method are used to generate the cerebral hemorrhage label image; The method of generating a cerebral hemorrhage label image based on the threshold segmentation method is to smooth the pre-processed 3D brain parenchyma CT image, and then perform global threshold segmentation on the smoothed image; Generating a cerebral hemorrhage label image based on a clustering segmentation method involves smoothing the preprocessed 3D brain parenchyma CT image and then performing fuzzy clustering segmentation on the smoothed image.

4. The multi-branch segmentation system for cerebral hemorrhage and peripheral edema based on automatically generated labels according to claim 3, characterized in that: In the label image generation module, the threshold segmentation method, cluster segmentation method and contralateral difference method are used to generate the peripheral edema label image; The threshold segmentation method is used to generate the surrounding edema label image. The cerebral hemorrhage label image of the same 3D cranial CT image is expanded outward according to a certain radius, and the initial edema area is determined by dot multiplication with the pre-processed 3D brain parenchyma CT image. The initial edema area is then segmented by a single threshold. The clustering segmentation method is used to generate the peripheral edema label image. The cerebral hemorrhage label image of the same 3D cranial CT image is expanded outward according to a certain radius, and the dot product is performed with the pre-processed 3D brain parenchyma CT image to determine the initial edema area. The initial edema area is then fuzzy clustered and segmented. The contralateral difference method is used to generate a peripheral edema label image. The cerebral hemorrhage label image of the same 3D cranial CT image is expanded outward according to a certain radius, and the initial edema area is determined by dot multiplication with the preprocessed 3D brain parenchyma CT image. Then, a mirror reversal is performed based on the midline to extract the corresponding area of ​​the contralateral brain. Finally, the two areas are subtracted, and the area with the difference within a certain range is retained as the target edema area.

5. The multi-branch segmentation system for cerebral hemorrhage and peripheral edema based on automatically generated labels according to claim 1, characterized in that: In the positioning network, the Dice similarity coefficient loss function is used; In the segmentation network, an average weighted or adaptively weighted Dice similarity coefficient loss function is used.

6. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by the processor, the following steps are implemented: Preprocessing the acquired 3D brain CT images; Computer vision and machine learning algorithms are used to automatically generate cerebral hemorrhage label images and surrounding edema label images from pre-processed 3D brain parenchyma CT images; Automatically generated labeled images of cerebral hemorrhage and surrounding edema are used to train a multi-branch segmentation model for cerebral hemorrhage and surrounding edema based on deep learning, thereby simultaneously segmenting both cerebral hemorrhage and surrounding edema. Using the deep learning segmentation results of cerebral hemorrhage and surrounding edema, morphological measurements are performed and the results are presented. The deep learning-based multi-branch segmentation model for cerebral hemorrhage and peripheral edema includes a segmentation network with an encoding-decoding U-shaped framework as the backbone network. The segmentation network includes an encoding branch and multiple decoding branches with or without shared parameters, and the number of decoding branches is ≥2. The segmentation network is trained using automatically generated labeled images of cerebral hemorrhage and surrounding edema. An attention mechanism module or a Transformer module is set at the output of each decoding branch to integrate the input pre-processed 3D brain parenchyma CT image information into the segmentation network. The deep learning-based multi-branch segmentation model for cerebral hemorrhage and peripheral edema further includes a localization network with an encoding-decoding U-shaped framework as a backbone network, wherein the localization network includes an encoding branch and a decoding branch; The automatically generated cerebral hemorrhage label image and surrounding edema label image are used to train the localization network, and the image output by the decoding branch is multiplied with the preprocessed 3D brain parenchyma CT image, and the resulting image is then input into the segmentation network to train the segmentation network.

7. The computer-readable storage medium according to claim 6, wherein: The preprocessing of the acquired 3D brain CT images includes: data cleaning, format conversion, renaming, steering, zero voxel filling, registration, and window width and window position adjustment of the acquired 3D brain CT images, and finally extracting the brain parenchyma image.

8. The computer-readable storage medium according to claim 6, wherein: The threshold segmentation method and clustering segmentation method are used to generate cerebral hemorrhage label images respectively; The method of generating a cerebral hemorrhage label image based on the threshold segmentation method is to smooth the pre-processed 3D brain parenchyma CT image, and then perform global threshold segmentation on the smoothed image; Generating a cerebral hemorrhage label image based on a clustering segmentation method involves smoothing the preprocessed 3D brain parenchyma CT image and then performing fuzzy clustering segmentation on the smoothed image.

9. The computer-readable storage medium according to claim 8, wherein: The peripheral edema label images were generated using the threshold segmentation method, clustering segmentation method and contralateral difference method respectively; The threshold segmentation method is used to generate the surrounding edema label image. The cerebral hemorrhage label image of the same 3D cranial CT image is expanded outward according to a certain radius, and the initial edema area is determined by dot multiplication with the pre-processed 3D brain parenchyma CT image. The initial edema area is then segmented by a single threshold. The clustering segmentation method is used to generate the peripheral edema label image. The cerebral hemorrhage label image of the same 3D cranial CT image is expanded outward according to a certain radius, and the dot product is performed with the pre-processed 3D brain parenchyma CT image to determine the initial edema area. The initial edema area is then fuzzy clustered and segmented. The contralateral difference method is used to generate a peripheral edema label image. The cerebral hemorrhage label image of the same 3D cranial CT image is expanded outward according to a certain radius, and the initial edema area is determined by dot multiplication with the preprocessed 3D brain parenchyma CT image. Then, a mirror reversal is performed based on the midline to extract the corresponding area of ​​the contralateral brain. Finally, the two areas are subtracted, and the area with the difference within a certain range is retained as the target edema area.

10. The computer-readable storage medium according to claim 6, wherein: In the positioning network, the Dice similarity coefficient loss function is used; In the segmentation network, the average weighted and adaptive weighted Dice similarity coefficient loss functions are used.

Images