Three-dimensional brain midline detection method and device, computer device and storage medium

Abstract

The application relates to a three-dimensional brain midline detection method and device, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring a brain scan image, pre-processing the brain scan image to obtain a training set image and a to-be-detected image; training a preset three-dimensional detection neural network based on the training set image to perform a left and right hemisphere segmentation task, and obtaining a hemisphere segmentation model; initializing the three-dimensional detection neural network based on network parameters of the hemisphere segmentation model, combining the training set image, training the three-dimensional detection neural network to perform a brain midline detection task, and obtaining a brain midline detection model; and detecting the input to-be-detected image based on the brain midline detection model to obtain a brain midline detection result. According to the application, the three-dimensional model training can be simultaneously performed in combination with the brain midline detection task and the left and right hemisphere segmentation task, the brain midline is determined according to the anatomical structure of the left and right hemispheres, and the problem that the brain midline detection accuracy is reduced when the brain midline structure is blocked is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a three-dimensional brain midline detection method and device, computer equipment and storage medium. BACKGROUND

[0002] In the medical field, the brain midline is an anatomical structure that separates the left and right hemispheres, and is usually located in the center of the brain on a non-traumatic brain image. When the ventricle and other structures are squeezed due to bleeding, the actual brain midline in the NCCT image (Non-Contrast Computer Tomography, CT scan) usually has a deviation. By calculating the actual brain midline position, the ideal brain midline position and the deviation between the two, the occupying effect and high intracranial pressure in the brain can be indicated, providing rich information for doctors to assess the severity of stroke or traumatic brain injury.

[0003] Currently, deep convolutional neural networks can extract feature information of the brain midline to segment brain structures, such as detecting the positions of the two endpoints of the brain midline or the coordinates of the brain midline to determine the actual position of the brain midline. However, for patients with intraventricular hemorrhage, the intraventricular hemorrhage produces a hematoma, which can block the brain midline structure and affect the detection of the brain midline. Therefore, when the brain midline structure is blocked, the accuracy of brain midline detection is reduced.

[0004] Currently, there is no effective solution to the problem of reduced accuracy of brain midline detection when the brain midline structure is blocked in related technologies. SUMMARY

[0005] A three-dimensional brain midline detection method, device, computer equipment and storage medium are provided in the present embodiment to solve the problem of reduced accuracy of brain midline detection when the brain midline structure is blocked in related technologies.

[0006] In a first aspect, a three-dimensional brain midline detection method is provided in the present embodiment, comprising:

[0007] Obtaining a brain scan image, pre-processing to obtain a training set image and a to-be-detected image;

[0008] Training a pre-set three-dimensional detection neural network based on the training set image for a left and right hemisphere segmentation task to obtain a hemisphere segmentation model;

[0009] Initializing the three-dimensional detection neural network based on the network parameters of the hemisphere segmentation model, and training the three-dimensional detection neural network for a brain midline detection task in combination with the training set image to obtain a brain midline detection model;

[0010] detecting the input image to obtain a brain midline detection result.

[0011] In some embodiments, before the detecting the input image to obtain a brain midline detection result, the method further comprises:

[0012] segmenting the input image to obtain an eyeball midpoint;

[0013] determining a correction angle of the input image based on the eyeball midpoint and the hemisphere segmentation model, and performing angle normalization processing on the input image according to the correction angle.

[0014] In some embodiments, the segmenting the input image to obtain an eyeball midpoint comprises:

[0015] annotating the left eyeball and the right eyeball in the training set image as labels of the training set image;

[0016] establishing a two-dimensional segmentation neural network, inputting the training set image and the labels of the left eyeball and the right eyeball into the two-dimensional segmentation neural network, and training to obtain the eyeball segmentation model;

[0017] detecting the input image based on the eyeball segmentation model to obtain a segmentation result of the left eyeball and the right eyeball, and processing to obtain the eyeball midpoint.

[0018] In some embodiments, the determining a correction angle of the input image based on the eyeball midpoint and the hemisphere segmentation model, and performing angle normalization processing on the input image according to the correction angle comprises:

[0019] detecting the input image based on the hemisphere segmentation model to obtain a segmentation result of the left hemisphere and the right hemisphere, and processing to obtain a centroid of brain tissue;

[0020] determining the correction angle according to a line connecting the eyeball midpoint and the centroid of brain tissue to generate a three-dimensional rigid transformation matrix for angle normalization processing on the input image.

[0021] In some embodiments, the obtaining brain scan images, pre-processing the brain scan images to obtain training set images and input images comprises:

[0022] obtaining the brain scan images, performing format conversion, and screening the brain scan images based on effectiveness, and performing normalization processing;

[0023] The training set image and the to-be-detected image are obtained by dividing the brain scan image after the preprocessing.

[0024] In some embodiments, the training of the preset three-dimensional detection neural network on a left-right hemisphere segmentation task based on the training set image obtains a hemisphere segmentation model, including:

[0025] The left hemisphere and the right hemisphere in the training set image are labeled as labels of the training set image.

[0026] The three-dimensional detection neural network is established, and the training set image and the labels of the left hemisphere and the right hemisphere are input into the three-dimensional detection neural network, so as to obtain the hemisphere segmentation model through training.

[0027] In some embodiments, the initialization of the three-dimensional detection neural network based on the network parameters of the hemisphere segmentation model, in combination with the training set image, trains the three-dimensional detection neural network on a brain midline detection task to obtain a brain midline detection model, including:

[0028] According to the labels of the left hemisphere and the right hemisphere, the position of the brain midline anatomical structure in the training set image is calculated and labeled as a label of the training set image.

[0029] The three-dimensional detection neural network is initialized based on the network parameters of the hemisphere segmentation model.

[0030] The training set image and the label of the brain midline are input into the initialized three-dimensional detection neural network, so as to obtain the brain midline detection model through training.

[0031] In a second aspect, a three-dimensional brain midline detection device is provided in the embodiments, including an image acquisition module, a three-dimensional model training module, and a brain midline detection module.

[0032] The image acquisition module is configured to acquire a brain scan image, and obtain a training set image and a to-be-detected image after preprocessing.

[0033] The three-dimensional model training module is configured to train a preset three-dimensional detection neural network on a left-right hemisphere segmentation task based on the training set image, and obtain a hemisphere segmentation model.

[0034] The three-dimensional detection neural network is initialized based on the network parameters of the hemisphere segmentation model, and the three-dimensional detection neural network is trained on a brain midline detection task in combination with the training set image, so as to obtain a brain midline detection model.

[0035] The brain midline detection module is used to detect the input image to be detected based on the brain midline detection model, and obtain the brain midline detection result.

[0036] Thirdly, this embodiment provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the three-dimensional brain midline detection method described in the first aspect above.

[0037] Fourthly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the three-dimensional brain midline detection method described in the first aspect above.

[0038] Compared with related technologies, the three-dimensional brain midline detection method, apparatus, computer equipment, and storage medium provided in this embodiment acquire brain scan images, preprocess them to obtain training set images and images to be detected; based on the training set images, a preset three-dimensional detection neural network is trained for left and right hemisphere segmentation tasks to obtain a hemisphere segmentation model; based on the network parameters of the hemisphere segmentation model, the three-dimensional detection neural network is initialized, and combined with the training set images, the three-dimensional detection neural network is trained for brain midline detection tasks to obtain a brain midline detection model; based on the brain midline detection model, the input images to be detected are detected to obtain brain midline detection results. This method can combine brain midline detection tasks and left and right hemisphere segmentation tasks to simultaneously train a three-dimensional model to obtain a brain midline detection model. In this way, the brain midline is determined based on the anatomical structure of the left and right hemispheres, which can improve the accuracy of brain midline detection in cases such as occlusion of brain midline structures due to intraventricular hemorrhage, and solve the problem of reduced accuracy of brain midline detection when brain midline structures are occluded.

[0039] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0040] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0041] Figure 1 This is a hardware structure block diagram of the terminal of the three-dimensional brain midline detection method in one embodiment;

[0042] Figure 2 This is a flowchart of a three-dimensional brain midline detection method in one embodiment;

[0043] Figure 3 This is a schematic diagram of the VBNet network structure used in one embodiment;

[0044] Figure 4 This is a flowchart of a preferred embodiment of a three-dimensional brain midline detection method;

[0045] Figure 5 This is a schematic diagram of the brain midline detection process in a preferred embodiment;

[0046] Figure 6 This is a schematic diagram of the model training process in a preferred embodiment;

[0047] Figure 7 This is a structural block diagram of a three-dimensional brain midline detection device in one embodiment.

[0048] In the diagram: 102, processor; 104, memory; 106, transmission device; 108, input / output device; 10, image acquisition module; 20, 3D model training module; 30, brain midline detection module. Detailed Implementation

[0049] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0050] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0051] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the three-dimensional brain midline detection method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0052] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the three-dimensional brain midline detection method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0053] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0054] In the medical field, the midline of the brain is the anatomical structure separating the left and right hemispheres, and it is usually located in the center of the brain on images without brain trauma. Non-contrast computer-to-texture (NCCT) imaging has become the first-line imaging method for acute stroke due to its wide application, short examination time, low cost, and ability to accurately detect subarachnoid hemorrhage and intracranial hemorrhage. Its scanning range extends from the bottom of the posterior fossa upwards to the highest point of the brain's convex surface, acquiring three-dimensional digital image data. When hemorrhage causes a mass effect in structures such as the ventricles, compressing the brain, the actual midline of the brain on NCCT images is usually offset. By calculating the actual midline position, the ideal midline position, and the offset between the two, it is possible to indicate the mass effect and increased intracranial pressure within the brain, providing doctors with rich information to assess the severity of stroke or traumatic brain injury.

[0055] Traditional midline detection methods mainly define key points in images by using anatomical information of the human brain. By detecting these key points, the structure of the midline on the image can be fitted. This can overcome the shortcomings of traditional methods that rely on doctors to manually delineate the brain midline, which have large subjective differences. However, this type of method cannot accurately fit the brain midline with high intensity deviation in images of hemorrhage patients.

[0056] Deep convolutional neural networks can extract feature information about the brain midline for brain structure segmentation, such as detecting the positions of the two endpoints of the midline or its coordinates to determine its actual location. However, in some patients with intraventricular hemorrhage, the hematoma caused by the hemorrhage can obstruct the midline structure, thus affecting its detection. Therefore, the accuracy of midline detection decreases when the midline structure is obstructed.

[0057] To address the above issues, the following embodiments provide a three-dimensional brain midline detection method, apparatus, computer device, and storage medium that can simultaneously train a three-dimensional model by combining brain midline detection tasks and left and right hemisphere segmentation tasks. This allows for determining the brain midline based on the anatomical structures of the left and right hemispheres, which can improve the accuracy of brain midline detection in situations such as when the brain midline structure is obscured due to intraventricular hemorrhage.

[0058] This embodiment provides a three-dimensional brain midline detection method. Figure 2 This is a flowchart of the three-dimensional brain midline detection method in this embodiment, as follows: Figure 2 As shown, the method includes the following steps:

[0059] Step S210: Obtain brain scan images, and after preprocessing, obtain training set images and images to be detected.

[0060] Specifically, brain scan images are acquired through CT (Computed Tomography) or MRI scans. These brain scan images include, but are not limited to, NCCT images, CT images, and MRI images. After preprocessing such as format conversion, checking the validity of the brain scan images, and normalization, the brain scan images can be divided into training set images and images to be tested according to a certain ratio.

[0061] Step S220: Based on the training set images, train the preset 3D detection neural network for left and right hemisphere segmentation tasks to obtain a hemisphere segmentation model.

[0062] Specifically, using medical image annotation tools, the left and right hemispheres in the training set images are labeled as tags for the training set images. Based on the training set images and the labels for the left and right hemispheres, a 3D detection neural network is trained to perform left and right hemisphere segmentation tasks, resulting in a hemisphere segmentation model. The pre-defined 3D detection neural network includes, but is not limited to, a 3D V-shaped encoder-decoder network structure, such as VBNet.

[0063] Step S230: Initialize the 3D detection neural network based on the network parameters of the hemispherical segmentation model, and train the 3D detection neural network for the brain midline detection task by combining the training set images to obtain the brain midline detection model.

[0064] Specifically, based on the network parameters of the hemispherical segmentation model trained in step S220 above, the parameters of the preset three-dimensional detection neural network are initialized, wherein the network parameters can be weight parameters in the network structure, etc.

[0065] Since the midline of the brain is the dividing line between the left and right hemispheres in anatomical structure, the location of the midline structure is obtained and labeled according to the left and right hemispheres marked in step S220. Based on the training set images and the midline label, the three-dimensional detection network model that has been initialized with parameters is trained for the midline detection task to obtain the midline detection model.

[0066] Step S240: Based on the brain midline detection model, the input image to be detected is detected to obtain the brain midline detection result.

[0067] Specifically, the preprocessed image to be detected is input into the brain midline detection model, and the brain midline detection result is output.

[0068] The above steps combine the brain midline detection task and the left and right hemisphere segmentation task to train a 3D model simultaneously, resulting in a brain midline detection model. Compared with existing technologies that detect the positions of the two endpoints of the brain midline or the brain midline coordinates to determine the actual location of the brain midline, this method of determining the brain midline based on the anatomical structure of the left and right hemispheres can improve the accuracy and robustness of brain midline detection in cases such as occlusion of the brain midline structure due to intraventricular hemorrhage, and solve the problem of reduced accuracy of brain midline detection when the brain midline structure is occluded.

[0069] Furthermore, by analyzing three-dimensional image data based on a three-dimensional detection neural network, and making full use of the spatial context information in three-dimensional brain scan images, compared with the existing technology that uses a 2D detection model to process and segment 2D cross-sectional images, the accuracy and robustness of the overall brain midline detection task can be improved. This embodiment is mainly applied to the brain midline segmentation task of NCCT three-dimensional images, but it can also be extended to the brain midline segmentation application of CT or MR modal images.

[0070] In some embodiments, the acquisition of brain scan images in step S210 above, followed by preprocessing to obtain training set images and images to be detected, can be achieved through the following steps:

[0071] Step S211: Obtain brain scan images, perform format conversion, filter brain scan images based on validity, and perform normalization processing.

[0072] Specifically, DICOM (Digital Imaging and Communications in Medicine) images obtained from CT scans are acquired, anonymized, and converted to the commonly used Nifty image format. Brain scan images that have passed validity screening are then normalized. Validity checks include verifying the correctness of the data format and the presence of abnormal data thresholds, as well as normalization using the window width and window level of the brain window modality. The following is a representation of a normalization formula:

[0073] x′=(x-WL) / WW;

[0074] Where x represents the normalized input, x' represents the normalized output; WL represents the window level; and WW represents the window width.

[0075] Step S212: By dividing the preprocessed brain scan images, training set images and images to be detected are obtained.

[0076] Specifically, different methods can be used to divide brain scan images into training set images and images to be tested. Generally, the division is random, and the training set and validation set are independent and do not overlap to ensure that no information leakage occurs.

[0077] In this embodiment, brain scan images are preprocessed to obtain training set images and images to be detected. The preprocessing includes validity screening and normalization, which can improve the quality of the training set images and images to be detected. In the normalization process, setting appropriate window width and window level makes the features in the images more obvious.

[0078] In some embodiments, step S220, based on the training set images, trains a preset 3D detection neural network for left and right hemisphere segmentation tasks to obtain a hemisphere segmentation model, which can be achieved through the following steps:

[0079] Step S221: Label the left and right hemispheres in the training set images as labels for the training set images.

[0080] Specifically, the labels for the left and right hemispheres can be manually obtained by professional doctors using medical image annotation tools based on their professional knowledge, and used as labels for the training set images.

[0081] Step S222: Establish a three-dimensional detection neural network by inputting the training set images and the labels of the left and right hemispheres into the three-dimensional detection neural network to train and obtain a hemisphere segmentation model.

[0082] Specifically, the 3D detection neural network built on VBNet is based on a V-type encoder-decoder network structure. Figure 3 This is a schematic diagram of the VBNet network structure used in this embodiment, as shown below. Figure 3 As shown, the network structure includes four downsampling layers, four upsampling layers, and four skip connection layers between the upsampling and downsampling layers. Each downsampling layer consists of several convolutional units and a downsampling operation, while each upsampling layer consists of several convolutional units and an upsampling operation. Specifically: the convolutional units have a stride of 1 and sequentially perform 3D convolution operations, non-linear activation operations, and batch normalization operations; the downsampling operation is performed as a convolutional unit with a stride of 2; the upsampling operation is performed as a deconvolutional unit with a stride of 2 (where the operator is a 3D deconvolution operator); the skip connection layers concatenate the input feature maps along the channel dimension.

[0083] The training set images and the labels for the left and right hemispheres are input into the aforementioned 3D detection neural network for training to obtain a hemisphere segmentation model. The training process includes the following steps:

[0084] (1) Normalize the training set images and perform appropriate image enhancement operations such as cropping and flipping. For example, for the above three-dimensional brain scan images, input them into the three-dimensional detection neural network model with an image size of 192×192×16.

[0085] (2) Using the annotations of the left and right hemispheres as the labels of the training set images, construct the VBNet learning function DoubleLoss, where DoubleLoss = DiceLoss + FocalLoss, and the specific calculation method is as follows:

[0086]

[0087]

[0088] in, Here, y represents the model's predicted value, and y represents the label; W i The weights for each segmentation category are α and γ, which are constants set to 0.25 and 2, respectively.

[0089] (3) The model is trained by combining the Adam optimizer with weight decay. Adam can automatically adjust the learning rate according to the learning progress of the model. Weight decay is set to reduce the weight when the model iterates to a certain percentage. It mainly prevents the model from overfitting and plays a certain role in regularization. When the model's learning function decreases to the point where it no longer decreases, the model stops training.

[0090] The hemisphere segmentation model obtained in this embodiment, based on the left and right hemisphere segmentation task, can segment the left and right hemispheres of the image to be detected and obtain a more accurate location of the brain midline anatomical structure from the positional relationship of the anatomical structure of the left and right hemispheres.

[0091] In some embodiments, step S230 above initializes the 3D detection neural network based on the network parameters of the hemispherical segmentation model, and trains the 3D detection neural network for the brain midline detection task using training set images to obtain a brain midline detection model. This can be achieved through the following steps:

[0092] Step S231: Based on the annotations of the left and right hemispheres, calculate the location of the brain midline anatomical structure in the training set images and annotate it as the label of the training set images.

[0093] Specifically, based on the anatomical structures of the left and right hemispheres, as well as the relationship between morphological expansion and intersection, the location of the anatomical structure along the brain midline is calculated, labeled, and used as labels for the training set images.

[0094] Step S232: Initialize the 3D detection neural network based on the network parameters of the hemispherical segmentation model.

[0095] Specifically, in the above embodiments, a hemisphere segmentation model is trained based on the left and right hemisphere segmentation task. The three-dimensional detection neural network established in the above embodiments is initialized according to the network parameters of the hemisphere segmentation model, such as weight parameters.

[0096] Step S233: Input the training set images and the corresponding labels of the brain midline into the initialized three-dimensional detection neural network to train and obtain the brain midline detection model.

[0097] Specifically, based on the initialized 3D detection neural network, training set images and brain midline labels are input, and a brain midline detection model is trained based on the brain midline detection task.

[0098] In this embodiment, the 3D detection neural network is initialized based on the network parameters of the hemispherical segmentation model, and then trained on the midline detection task to obtain the midline detection model. This allows for simultaneous 3D model training combining the midline detection task and the left and right hemisphere segmentation task, resulting in a midline detection model. By determining the midline based on the anatomical structure of the left and right hemispheres, the positions of the midline and the left and right hemispheres are mutually adaptive, improving the accuracy of midline detection. Furthermore, by analyzing 3D image data based on the 3D detection neural network, fully utilizing the spatial context information in the 3D brain scan images, compared to the existing technology that uses a 2D detection model to process and segment 2D cross-sectional images, the overall accuracy and robustness of the midline detection task can be improved.

[0099] Because clinical diagnosis requires the rapid acquisition of brain scans, standardized positioning is often impossible during scanning, resulting in large tilt angles in the acquired brain scans, which affects subsequent detection of midline brain structures. Based on the above embodiments, the following embodiments provide steps for angle standardization processing of the images to be tested.

[0100] In some embodiments, before obtaining the brain midline detection result based on the brain midline detection model in step S240, the image to be detected may be subjected to angle normalization processing through the following steps:

[0101] The image to be detected is segmented into left and right eyeballs to obtain the midpoint of the eyeballs; the correction angle of the image to be detected is determined based on the midpoint of the eyeballs and the hemisphere segmentation model; and the image to be detected is then subjected to angle standardization processing according to the correction angle.

[0102] Specifically, the positions of the left and right eyeballs are determined by segmenting the left and right eyeballs, and then the midpoint of the left and right eyeballs is obtained. The segmentation of the left and right eyeballs in the image to be detected can be achieved through eyeball segmentation models or manual measurement and segmentation.

[0103] The centroid of brain tissue is determined by a hemispherical segmentation model, and the corresponding correction angle of the image to be detected is determined by combining the midpoint of the eyeball. The tilt angle of the image to be detected is adjusted according to the calibration angle to achieve angle standardization processing.

[0104] In this embodiment, by segmenting the image to be detected into left and right eyeballs to obtain the midpoint of the eyeballs, and then determining the centroid of the brain tissue based on the hemispherical segmentation model, the correction angle can be determined according to the positional angular relationship between the midpoint of the eyeballs and the centroid of the brain tissue, thereby achieving angle standardization processing of the image to be detected and reducing the impact on the accuracy of midline brain structure detection.

[0105] In some embodiments, the above-described method of segmenting the image to be detected into left and right eyeballs to obtain the midpoint of the eyeballs includes the following steps:

[0106] The left and right eyeballs in the training set images are labeled and used as the labels for the training set images;

[0107] A two-dimensional segmentation neural network is established. The training set images and the labels of the left and right eyeballs are input into the two-dimensional segmentation neural network to train the eyeball segmentation model.

[0108] Based on the eye segmentation model, the input image to be detected is processed to obtain the segmentation results of the left and right eyeballs, and the midpoint of the eyeball is obtained.

[0109] Specifically, this embodiment introduces an eye segmentation model to perform angle normalization processing on the image to be detected. Specifically, a two-dimensional cross-sectional image including the eye positions is obtained from the training set images. Using a medical image annotation tool, the left and right eyeballs in the two-dimensional cross-sectional image are labeled as tags for the training set images.

[0110] A two-dimensional segmentation neural network is built based on UBNet, where the network structure of UBNet is similar to that of VBNet, as specifically described in the above embodiments. Figure 3 Based on the VBNet network structure, the overall architecture is kept unchanged, and the 3D convolution operator is replaced with the 2D convolution operator.

[0111] The training set images, along with the labels for the left and right eyeballs, are input into the aforementioned two-dimensional segmentation neural network for training, resulting in an eyeball segmentation model. The training process is similar to that of the hemispherical segmentation model in the above embodiment, except that normalization is performed using different window widths and window levels to make the eyeball features more prominent. The training set images are then cropped and input into the two-dimensional segmentation neural network model at a size of 192×192. The learning function of UBNet is DiceLoss.

[0112] The segmentation results of the left and right eyeballs are obtained through an eyeball segmentation model, and the midpoint of the left and right eyeballs is obtained based on their positions.

[0113] The eye segmentation model obtained by training based on the eye segmentation task in this embodiment is easy to train and can accurately segment the left and right eyeballs in the image to be detected, and then calculate the correction angle based on the midpoint of the left and right eyeballs in the subsequent process.

[0114] In some embodiments, the above-described method for determining the correction angle of the image to be detected based on the midpoint of the eyeball and a hemispherical segmentation model, and then performing angle standardization processing on the image to be detected according to the correction angle, includes the following steps:

[0115] Based on the hemispherical segmentation model, the input image to be detected is detected to obtain the segmentation results of the left and right hemispheres, and the centroid of the brain tissue is obtained through processing.

[0116] The correction angle is determined by connecting the midpoint of the eyeball and the centroid of the brain tissue, and a three-dimensional rigid transformation matrix is ​​generated to perform angle standardization processing on the image to be detected.

[0117] Specifically, the segmentation results of the left and right hemispheres are obtained through a hemispherical segmentation model. The segmentation results are then binarized, and the foreground region is extracted to obtain the brain tissue after merging the left and right hemispheres. The centroid of the brain tissue is then calculated.

[0118] Using the horizontal and vertical axes of the image to be detected as the basic coordinate system, a line is established connecting the midpoint of the eyeball and the centroid of the brain tissue. The correction angle is calculated by combining the vertical axis and the vertical axis in the basic coordinate system. An angle correction is performed on the image to be detected by generating a three-dimensional rigid transformation matrix.

[0119] This embodiment introduces an eye segmentation model to obtain the midpoints of the left and right eyeballs, and combines the centroid of brain tissue to calculate and correct the angle. Compared with the current methods that rely on the accuracy of coarse segmentation of the brain midline or angle correction methods that use auxiliary tasks to train the model together, this method is more in line with clinical practice. Furthermore, the application of rigid transformation for angle standardization can avoid image distortion problems caused by non-rigid transformation.

[0120] The present embodiment will now be described and illustrated through preferred embodiments.

[0121] Figure 4 This is a flowchart of the three-dimensional brain midline detection method according to a preferred embodiment, as follows: Figure 4 As shown, the method includes the following steps:

[0122] Step S410: Obtain brain scan images. After preprocessing, divide the images into training set images and images to be detected. Label the left eyeball, right eyeball, left hemisphere, right hemisphere, and the brain midline calculated based on the left and right hemispheres in the training set images as labels for the training set images.

[0123] Step S420: Establish a two-dimensional segmentation neural network by inputting the training set images and the labels of the left and right eyeballs into the two-dimensional segmentation neural network, and obtain the eyeball segmentation model after training.

[0124] Step S430: Establish a three-dimensional detection neural network by inputting the training set images and the labels of the left and right hemispheres into the three-dimensional detection neural network, and obtain a hemisphere segmentation model after training.

[0125] Step S440: Initialize the 3D detection neural network based on the network parameters of the hemispherical segmentation model; input the training set images and brain midline labels into the initialized 3D detection neural network, and obtain the brain midline detection model after training.

[0126] Step S450: For the input image to be detected, based on the eye segmentation model, the segmentation results of the left and right eyeballs are obtained, and the midpoint of the eyeballs is obtained; based on the hemispherical segmentation model, the segmentation results of the left and right hemispheres are obtained, and the centroid of the brain tissue is obtained; based on the line connecting the midpoint of the eyeballs and the centroid of the brain tissue, the correction angle is obtained to generate a three-dimensional rigid transformation matrix to perform angle correction on the image to be detected.

[0127] Step S460: Input the angle-corrected image to be detected into the brain midline detection model to obtain the brain midline detection result.

[0128] It should be noted that the steps shown in the above process or in the flowcharts of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here. For example, step S420, which trains the eye segmentation model, may be executed between steps S410 and S450.

[0129] Figure 5 This is a schematic diagram of the brain midline detection process in this preferred embodiment, as shown below. Figure 5 As shown, in the data processing stage, brain scan images are acquired and preprocessed. Labels are obtained by annotating the left eyeball, right eyeball, left hemisphere, right hemisphere, and the brain midline calculated based on the left and right hemispheres. Then, network training is performed based on the labels and the acquired brain scan images.

[0130] In the brain midline detection stage, the image to be detected is input into the eye segmentation model to obtain the segmentation results of the left and right eyeballs, and the coordinates of the eyeball midpoint are calculated. The image to be detected is input into the hemisphere segmentation model to obtain the segmentation results of the left and right hemispheres, and the brain tissue centroid coordinates are calculated after merging. The eyeball midpoint coordinates and brain tissue centroid coordinates are connected and compared with the base coordinate system of the image to be detected to obtain the correction angle, and the image to be detected is angle corrected.

[0131] Angle-corrected images of the target brain are input into a brain midline detection model to obtain the actual brain midline detection result. The brain midline offset is obtained by comparing it with the position of the ideal brain midline, and then the brain midline offset is used for clinical diagnosis.

[0132] Figure 6 This is a schematic diagram of the model training process in this preferred embodiment, as shown below. Figure 6 As shown, during the training process of the eye segmentation model, the input image is a training set image of size 192×192. In 2D-UBNet (two-dimensional segmentation neural network), the model is trained based on the labels of the left and right eyeballs, combined with the learning function DiceLoss.

[0133] During the training of the hemispherical segmentation model, the input image is a training set image of size 192×192×16. In 3D-VBNet (3D Detection Neural Network), the learning is performed based on the labels of the left and right hemispheres, combined with the learning function DoubleLoss.

[0134] During the training of the brain midline detection model, the input image is a training set image of size 192×192×16. After initializing 3D-VBNet based on the network parameters of the hemispherical segmentation model, it is trained according to the label of the brain midline.

[0135] This preferred embodiment addresses the problem that large tilt angles in clinically obtained brain scan images affect subsequent detection of the brain's midline structures. It introduces an eye segmentation model to obtain the midpoints of the left and right eyeballs, and combines this with the centroid calculation of brain tissue to correct the angle. Compared to current methods that rely on the accuracy of coarse midline segmentation or use auxiliary tasks to train the model together, this method is more in line with clinical practice. Furthermore, the application of rigid transformation for angle standardization avoids image distortion caused by non-rigid transformation.

[0136] Simultaneously, by combining the brain midline detection task and the left and right hemisphere segmentation task, a 3D model is trained to obtain a brain midline detection model. Compared with the existing technology that detects the positions of the two endpoints of the brain midline or the brain midline coordinates to determine the actual position of the brain midline, this method determines the brain midline based on the anatomical structure of the left and right hemispheres. This can improve the accuracy and robustness of brain midline detection in cases such as occlusion of brain midline structures due to intraventricular hemorrhage, and solve the problem of reduced accuracy of brain midline detection when brain midline structures are occluded.

[0137] Furthermore, by analyzing three-dimensional image data based on a three-dimensional detection neural network, and making full use of the spatial context information in three-dimensional brain scan images, compared with the existing technology that uses a 2D detection model to process and segment 2D cross-sectional images, the accuracy and robustness of the overall brain midline detection task can be improved. This embodiment is mainly applied to the brain midline segmentation task of NCCT three-dimensional images, but it can also be extended to the brain midline segmentation application of CT or MR modal images.

[0138] This embodiment also provides a three-dimensional brain midline detection device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as described previously. The terms "module," "unit," "subunit," etc., used below can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0139] Figure 7 This is a structural block diagram of the three-dimensional brain midline detection device in this embodiment, as shown below. Figure 7 As shown, the device includes: an image acquisition module 10, a three-dimensional model training module 20, and a brain midline detection module 30.

[0140] Image acquisition module 10 is used to acquire brain scan images, and after preprocessing, to obtain training set images and images to be detected.

[0141] The 3D model training module 20 is used to train a preset 3D detection neural network for left and right hemisphere segmentation tasks based on training set images to obtain a hemisphere segmentation model; the 3D detection neural network is initialized based on the network parameters of the hemisphere segmentation model, and then trained for brain midline detection tasks based on the training set images to obtain a brain midline detection model.

[0142] The brain midline detection module 30 is used to detect the input image to be detected based on the brain midline detection model and obtain the brain midline detection result.

[0143] The device provided in this embodiment combines the brain midline detection task and the left and right hemisphere segmentation task to simultaneously train a three-dimensional model, thereby obtaining a brain midline detection model. Compared with the existing technology that detects the positions of the two endpoints of the brain midline or the brain midline coordinates to determine the actual position of the brain midline, this method determines the brain midline based on the anatomical structure of the left and right hemispheres. This can improve the accuracy and robustness of brain midline detection in cases such as occlusion of the brain midline structure due to intraventricular hemorrhage, and solves the problem of reduced accuracy of brain midline detection when the brain midline structure is occluded.

[0144] Furthermore, by analyzing three-dimensional image data based on a three-dimensional detection neural network, and making full use of the spatial context information in three-dimensional brain scan images, compared with the existing technology that uses a 2D detection model to process and segment 2D cross-sectional images, the accuracy and robustness of the overall brain midline detection task can be improved. This embodiment is mainly applied to the brain midline segmentation task of NCCT three-dimensional images, but it can also be extended to the brain midline segmentation application of CT or MR modal images.

[0145] In some embodiments, the image acquisition module 10 is further configured to acquire brain scan images, perform format conversion, filter brain scan images based on validity, and perform normalization processing; by dividing the preprocessed brain scan images, training set images and images to be detected are obtained.

[0146] In some embodiments, the aforementioned 3D model training module 20 is further used to label the left and right hemispheres in the training set images as labels for the training set images; to establish a 3D detection neural network, and to input the training set images and the labels of the left and right hemispheres into the 3D detection neural network to train and obtain a hemisphere segmentation model.

[0147] In some embodiments, the brain midline detection module 30 is further configured to calculate the location of the brain midline anatomical structure in the training set images based on the annotations of the left and right hemispheres, and annotate it as the label of the training set images; initialize the three-dimensional detection neural network based on the network parameters of the hemisphere segmentation model; and input the corresponding labels of the training set images and the brain midline into the initialized three-dimensional detection neural network to train the brain midline detection model.

[0148] In some embodiments, the above-described apparatus further includes an angle correction module for segmenting the image to be detected into left and right eyeballs to obtain the midpoint of the eyeballs; determining the correction angle of the image to be detected based on the midpoint of the eyeballs and the hemispherical segmentation model; and performing angle standardization processing on the image to be detected according to the correction angle.

[0149] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0150] This embodiment also provides a computer device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0151] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0152] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0153] Furthermore, in conjunction with the three-dimensional brain midline detection method provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the three-dimensional brain midline detection methods described in the above embodiments.

[0154] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0155] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0156] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0157] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or alternative to other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0158] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A three-dimensional brain midline detection method, characterized in that, include: Brain scan images are acquired and preprocessed to obtain training set images and images to be detected; Based on the training set images, a preset 3D detection neural network is trained to perform left and right hemisphere segmentation tasks to obtain a hemisphere segmentation model. The three-dimensional detection neural network is initialized based on the network parameters of the hemispherical segmentation model. Combined with the training set images, the three-dimensional detection neural network is trained for the brain midline detection task to obtain the brain midline detection model. Based on the brain midline detection model, the input image to be detected is detected to obtain the brain midline detection result; Before obtaining the brain midline detection result by detecting the input image based on the brain midline detection model, the method further includes: The left and right eyeballs in the training set images are labeled as tags for the training set images; A two-dimensional segmentation neural network is established, and the training set images and the labels of the left eyeball and the right eyeball are input into the two-dimensional segmentation neural network to train and obtain an eyeball segmentation model; Based on the eye segmentation model, the input image to be detected is detected to obtain the segmentation results of the left and right eyeballs, and the midpoint of the eyeball is obtained through processing. Based on the hemispherical segmentation model, the input image to be detected is detected to obtain the segmentation results of the left and right hemispheres, and the centroid of the brain tissue is obtained through processing. Based on the line connecting the midpoint of the eyeball and the centroid of the brain tissue, the correction angle of the image to be detected is determined, so as to generate a three-dimensional rigid transformation matrix to perform angle standardization processing on the image to be detected. The 3D detection neural network is initialized based on the network parameters of the hemispherical segmentation model. Then, using the training set images, the 3D detection neural network is trained for a brain midline detection task to obtain a brain midline detection model, including: Based on the annotations of the left and right hemispheres in the training set images, the locations of the anatomical structures along the midline of the brain in the training set images are calculated and annotated as labels for the training set images; The three-dimensional detection neural network is initialized based on the network parameters of the hemispherical segmentation model. The training set images and the corresponding labels of the brain midline are input into the initialized three-dimensional detection neural network to train the brain midline detection model.

2. The three-dimensional brain midline detection method according to claim 1, characterized in that, The acquisition of brain scan images, after preprocessing, yields training set images and images to be detected, including: The brain scan images are acquired, their format is converted, and the brain scan images are filtered based on their validity and then normalized. The training set images and the images to be detected are obtained by dividing the preprocessed brain scan images.

3. The three-dimensional brain midline detection method according to claim 1, characterized in that, The step of training a preset 3D detection neural network for left and right hemisphere segmentation based on the training set images to obtain a hemisphere segmentation model includes: The left and right hemispheres in the training set images are labeled as tags for the training set images; The three-dimensional detection neural network is established by inputting the training set images and the labels of the left and right hemispheres into the three-dimensional detection neural network to train the hemisphere segmentation model.

4. A three-dimensional brain midline detection device, characterized in that, include: Image acquisition module, 3D model training module, angle correction module, and brain midline detection module; The image acquisition module is used to acquire brain scan images, and after preprocessing, obtain training set images and images to be detected; The 3D model training module is used to train a preset 3D detection neural network for left and right hemisphere segmentation based on the training set images, so as to obtain a hemisphere segmentation model. The three-dimensional detection neural network is initialized based on the network parameters of the hemispherical segmentation model. Combined with the training set images, the three-dimensional detection neural network is trained for the brain midline detection task to obtain the brain midline detection model. The brain midline detection module is used to detect the input image to be detected based on the brain midline detection model, and obtain the brain midline detection result; The angle correction module is used to label the left and right eyeballs in the training set images as labels for the training set images; A two-dimensional segmentation neural network is established. The training set images and the labels of the left and right eyeballs are input into the two-dimensional segmentation neural network to train an eyeball segmentation model. Based on the eyeball segmentation model, the input image to be detected is detected to obtain the segmentation results of the left and right eyeballs, and the midpoint of the eyeball is obtained. Based on the hemispherical segmentation model, the input image to be detected is detected to obtain the segmentation results of the left and right hemispheres, and the centroid of the brain tissue is obtained. According to the line connecting the midpoint of the eyeball and the centroid of the brain tissue, the correction angle of the image to be detected is determined to generate a three-dimensional rigid transformation matrix to perform angle standardization processing on the image to be detected. The brain midline detection module is further configured to calculate the location of the brain midline anatomical structure in the training set images based on the annotations of the left and right hemispheres in the training set images, and annotate it as the label of the training set images; initialize the three-dimensional detection neural network based on the network parameters of the hemisphere segmentation model; and input the corresponding labels of the training set images and the brain midline into the initialized three-dimensional detection neural network to train the brain midline detection model.

5. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the three-dimensional brain midline detection method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the three-dimensional brain midline detection method according to any one of claims 1 to 3.

Images