Brain midline identification method, device and equipment and storage medium

Abstract

The application provides a brain midline recognition method, device and equipment and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: acquiring a three-dimensional brain image; identifying a contour surface of a brain region from the three-dimensional brain image; performing image segmentation on the three-dimensional brain image to obtain a first half-brain region and a second half-brain region; identifying a first half-brain contour surface of the first half-brain region and a second half-brain contour surface of the second half-brain region from the three-dimensional brain image; and determining a difference set between a union of the first half-brain contour surface and the second half-brain contour surface and the contour surface of the brain region as a three-dimensional brain midline of the brain region. The above method uses the three-dimensional brain image to obtain the first half-brain contour surface and the second half-brain contour surface, and the difference set between the union of the first half-brain contour surface and the second half-brain contour surface and the contour surface of the brain region is the three-dimensional brain midline. The three-dimensional brain midline can provide more information and is more conducive to determining the brain state of a user.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a brain midline recognition method, apparatus, device and storage medium. Background Art

[0002] The brain midline is an anatomical structure that separates the left and right hemispheres of the brain. Determining the brain midline is important for judging the user's brain state and, in turn, measuring brain-related diseases. For example, in surgical navigation and puncture surgery scenarios, the brain midline can help doctors determine the position of the puncture needle relative to the brain midline, thereby effectively preventing the puncture needle from passing from one ventricle to the other. Related technologies usually identify the brain midline based on two-dimensional brain images. However, the brain midline identified in this way is a curve, which provides little information and is not conducive to determining the user's brain state. Summary of the Invention

[0003] The embodiments of the present application provide a brain midline recognition method, apparatus, device, and storage medium that can recognize the three-dimensional brain midline, thereby providing more information and facilitating the determination of the user's brain state. The technical solution is as follows:

[0004] In one aspect, a method for identifying a brain midline is provided, the method comprising:

[0005] acquiring a three-dimensional brain image, wherein the three-dimensional brain image includes a brain region;

[0006] identifying a contour surface of the brain region from the three-dimensional brain image;

[0007] Performing image segmentation on the three-dimensional brain image to obtain a first hemisphere region and a second hemisphere region;

[0008] identifying a first hemisphere outline surface of the first hemisphere region and a second hemisphere outline surface of the second hemisphere region from the three-dimensional brain image;

[0009] The difference between the union of the first half brain contour surface and the second half brain contour surface and the contour surface of the brain region is determined as the three-dimensional brain midline of the brain region.

[0010] In one possible implementation, determining the state corresponding to the brain region based on the three-dimensional brain image includes:

[0011] A classification model is called to classify the three-dimensional brain image to obtain a classification label, where the classification label is used to describe a state corresponding to the brain region, where the state is whether the lesion has broken into the ventricle or the lesion has not broken into the ventricle.

[0012] In one possible implementation, the training process of the classification model includes:

[0013] Acquiring second sample data, the second sample data including a sample three-dimensional brain image and a sample label, the sample three-dimensional brain image including a brain region, the sample label being used to describe a state corresponding to the brain region, the state being whether the lesion has ruptured into the ventricle or the lesion has not ruptured into the ventricle;

[0014] Calling the classification model to classify the sample three-dimensional brain image to obtain a predicted label;

[0015] The classification model is trained based on the sample labels and the predicted labels.

[0016] In a possible implementation, performing image segmentation on the three-dimensional brain image to obtain the lesion area includes:

[0017] The lesion segmentation model is called to perform image segmentation on the three-dimensional brain image to obtain the lesion area.

[0018] In a possible implementation, calling a lesion segmentation model to perform image segmentation on the three-dimensional brain image to obtain the lesion area includes:

[0019] calling the lesion segmentation model to perform image segmentation on the three-dimensional brain image to obtain a lesion probability map, wherein the lesion probability map is used to represent the probability that each point in the three-dimensional brain image belongs to a lesion area;

[0020] In the lesion probability map, points corresponding to probabilities greater than a second probability threshold are determined as points in the lesion area, and a set of points in the lesion area is determined as the lesion area.

[0021] In a possible implementation, the lesion segmentation model includes an encoding sub-model and a decoding sub-model, the encoding sub-model includes k encoding networks, and the decoding sub-model includes k decoding networks;

[0022] The calling of the lesion segmentation model to perform image segmentation on the three-dimensional brain image to obtain a lesion probability map includes:

[0023] Inputting the three-dimensional brain image into a first encoding network, performing feature extraction on the three-dimensional brain image based on the first encoding network, and outputting a first encoding feature map;

[0024] Based on the i-th encoding network, feature extraction is performed on the encoding feature map output by the previous network, and the i-th encoding feature map is output, until the k-th encoding network outputs the k-th encoding feature map, and the k-th encoding feature map is input into the first decoding network;

[0025] Reconstructing the k-th encoding feature map based on the first decoding network to obtain a first decoding feature map;

[0026] Based on the i-th decoding network, the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map are fused to perform image reconstruction, and the i-th decoding feature map is output, until the k-th decoding network outputs the k-th decoding feature map, and the k-th decoding feature map is determined as the lesion probability map;

[0027] Wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

[0028] In a possible implementation, the encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks; and the step of extracting features from the encoding feature map output by the previous network based on the i-th encoding network and outputting the i-th encoding feature map includes:

[0029] Based on the i-1th downsampling network, downsampling the encoding feature map output by the i-1th encoding network, outputting the i-1th intermediate encoding feature map, inputting the i-1th intermediate encoding feature map into the i-th encoding network, performing feature extraction on the i-1th intermediate encoding feature map based on the i-th encoding network, and outputting the i-th encoding feature map;

[0030] The method comprises: reconstructing an image of a feature map obtained by fusing a decoding feature map output by a previous network and a (k+1-i)th encoding feature map based on the i-th decoding network, and outputting the i-th decoding feature map.

[0031] Based on the i-1th upsampling network, the decoding feature map output by the i-1th decoding network is upsampled, and the i-1th intermediate decoding feature map is output. The i-1th intermediate decoding feature map is input into the i-th decoding network. Based on the i-th decoding network, the feature map after the fusion of the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map is reconstructed, and the i-th decoding feature map is output.

[0032] In one possible implementation, the training process of the lesion segmentation model includes:

[0033] Acquire third sample data, the third sample data including a sample three-dimensional brain image and a second sample segmented image, wherein the second sample segmented image is marked with a lesion area in the sample three-dimensional brain image;

[0034] calling the lesion segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain a lesion probability map, wherein the lesion probability map is used to represent the probability that each point in the sample three-dimensional brain image belongs to a lesion area;

[0035] The lesion segmentation model is trained based on the lesion probability map and the second sample segmentation image.

[0036] In a possible implementation, the training of the lesion segmentation model based on the lesion probability map and the second sample segmentation image includes:

[0037] determining at least one point belonging to a lesion area based on the second sample segmented image;

[0038] Determining, based on the lesion probability map, a probability that the at least one point belonging to the lesion area belongs to the lesion area;

[0039] Determining a loss value of the lesion segmentation model based on a probability that the at least one point belonging to the lesion area belongs to the lesion area;

[0040] Based on the loss value, the lesion segmentation model is trained.

[0041] In a possible implementation, the lesion segmentation model includes an encoding sub-model and a decoding sub-model, the encoding sub-model includes k encoding networks, and the decoding sub-model includes k decoding networks;

[0042] The calling of the lesion segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain a lesion probability map includes:

[0043] Inputting the sample three-dimensional brain image into a first encoding network, performing feature extraction on the sample three-dimensional brain image based on the first encoding network, and outputting a first encoding feature map;

[0044] Based on the i-th encoding network, feature extraction is performed on the encoding feature map output by the previous network, and the i-th encoding feature map is output, until the k-th encoding network outputs the k-th encoding feature map, and the k-th encoding feature map is input into the first decoding network;

[0045] Reconstructing the k-th encoding feature map based on the first decoding network to obtain a first decoding feature map;

[0046] Based on the i-th decoding network, the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map are fused to perform image reconstruction, and the i-th decoding feature map is output, until the k-th decoding network outputs the k-th decoding feature map, and the k-th decoding feature map is determined as the lesion probability map;

[0047] Wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

[0048] In a possible implementation, the encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks; and the step of extracting features from the encoding feature map output by the previous network based on the i-th encoding network and outputting the i-th encoding feature map includes:

[0049] Based on the i-1th downsampling network, downsampling the encoding feature map output by the i-1th encoding network, outputting the i-1th intermediate encoding feature map, inputting the i-1th intermediate encoding feature map into the i-th encoding network, performing feature extraction on the i-1th intermediate encoding feature map based on the i-th encoding network, and outputting the i-th encoding feature map;

[0050] The method comprises: reconstructing an image of a feature map obtained by fusing a decoding feature map output by a previous network and a (k+1-i)th encoding feature map based on the i-th decoding network, and outputting the i-th decoding feature map.

[0051] Based on the i-1th upsampling network, the decoding feature map output by the i-1th decoding network is upsampled, and the i-1th intermediate decoding feature map is output. The i-1th intermediate decoding feature map is input into the i-th decoding network. Based on the i-th decoding network, the feature map after the fusion of the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map is reconstructed, and the i-th decoding feature map is output.

[0052] In a possible implementation, acquiring a three-dimensional brain image includes:

[0053] Acquiring an original brain image, wherein the original brain image is a three-dimensional image;

[0054] Acquiring a template brain image, where the template brain image is bilaterally symmetrical;

[0055] According to the template brain image, the original brain image is rigidly registered to obtain the three-dimensional brain image.

[0056] In a possible implementation, acquiring a three-dimensional brain image includes:

[0057] Acquiring a computed tomography (CT) image of the brain, determining the window width and window level of the CT image as a reference window width and a reference window level, respectively, to obtain the three-dimensional brain image;

[0058] The window width represents the range of CT values ​​displayed in the CT image, and the window level represents the middle value of the range of CT values.

[0059] In a possible implementation, acquiring a three-dimensional brain image includes:

[0060] Acquiring an original brain image, wherein the original brain image includes multiple scanning layers;

[0061] Sampling the plurality of scanning layers so that an interval between any two scanning layers reaches a reference interval;

[0062] The sampled original brain image is used as the three-dimensional brain image.

[0063] In a possible implementation, acquiring a three-dimensional brain image includes:

[0064] Acquiring an original brain image, wherein the original brain image is a three-dimensional image;

[0065] The size of the original brain image is adjusted to a reference size to obtain the three-dimensional brain image.

[0066] In another aspect, a device for identifying a brain midline is provided, the device comprising:

[0067] an image acquisition module, configured to acquire a three-dimensional brain image, wherein the three-dimensional brain image includes a brain region;

[0068] a first recognition module, configured to recognize a contour surface of the brain region from the three-dimensional brain image;

[0069] an image segmentation module, configured to segment the three-dimensional brain image to obtain a first hemisphere region and a second hemisphere region;

[0070] a second recognition module, configured to recognize, from the three-dimensional brain image, a first hemisphere outline surface of the first hemisphere region and a second hemisphere outline surface of the second hemisphere region;

[0071] The brain midline determination module is used to determine the difference between the union of the first half brain contour surface and the second half brain contour surface and the contour surface of the brain region as the three-dimensional brain midline of the brain region.

[0072] In a possible implementation, the image segmentation module is configured to call a brain segmentation model to perform image segmentation on the three-dimensional brain image to obtain the first hemisphere region and the second hemisphere region.

[0073] In a possible implementation, the image segmentation module includes:

[0074] a first segmentation unit, configured to call the brain segmentation model and perform image segmentation on the three-dimensional brain image to obtain a first brain hemisphere probability map and a second brain hemisphere probability map, wherein the first brain hemisphere probability map is used to represent the probability that each point in the three-dimensional brain image belongs to the first brain hemisphere region, and the second brain hemisphere probability map is used to represent the probability that each point in the three-dimensional brain image belongs to the second brain hemisphere region;

[0075] The region determination unit is configured to determine, in the first hemisphere probability map, points whose corresponding probabilities are greater than a first probability threshold as points in the first hemisphere region, and to determine the set consisting of the points in the first hemisphere region as the first hemisphere region; and to determine, in the second hemisphere probability map, points whose corresponding probabilities are greater than the first probability threshold as points in the second hemisphere region, and to determine the set consisting of the points in the second hemisphere region as the second hemisphere region.

[0076] In one possible implementation, the brain segmentation model includes an encoding sub-model and a decoding sub-model, the encoding sub-model includes k encoding networks, and the decoding sub-model includes k decoding networks;

[0077] The first segmentation unit is configured to input the three-dimensional brain image into the first encoding network, perform feature extraction on the three-dimensional brain image based on the first encoding network, and output the first encoding feature map; perform feature extraction on the encoding feature map output by the previous network based on the i-th encoding network, output the i-th encoding feature map, until the k-th encoding network outputs the k-th encoding feature map, and input the k-th encoding feature map into the first decoding network; perform image reconstruction on the k-th encoding feature map based on the first decoding network to obtain the first decoding feature map; perform image reconstruction on the feature map after fusion of the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map based on the i-th decoding network, output the i-th decoding feature map, until the k-th decoding network outputs the k-th decoding feature map, and the k-th decoding feature map includes the first half-brain probability map and the second half-brain probability map; wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

[0078] In a possible implementation, the encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks;

[0079] The first segmentation unit is used to downsample the encoding feature map output by the i-1th encoding network based on the i-1th downsampling network, output the i-1th intermediate encoding feature map, input the i-1th intermediate encoding feature map to the i-th encoding network, perform feature extraction on the i-1th intermediate encoding feature map based on the i-th encoding network, and output the i-th encoding feature map; upsample the decoding feature map output by the i-1th decoding network based on the i-1th upsampling network, output the i-1th intermediate decoding feature map, input the i-1th intermediate decoding feature map to the i-th decoding network, perform image reconstruction on the feature map after fusion of the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map based on the i-th decoding network, and output the i-th decoding feature map.

[0080] In one possible implementation, the training process of the brain segmentation model includes:

[0081] Acquire first sample data, the first sample data including a sample three-dimensional brain image and a first sample segmented image, wherein the first sample segmented image is annotated with a first hemisphere region and a second hemisphere region in the sample three-dimensional brain image;

[0082] Invoking the brain segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain a first half-brain probability map and a second half-brain probability map, wherein the first half-brain probability map is used to represent the probability that each point in the sample three-dimensional brain image belongs to the first half-brain region, and the second half-brain probability map is used to represent the probability that each point in the sample three-dimensional brain image belongs to the second half-brain region;

[0083] The brain segmentation model is trained based on the first brain hemisphere probability map, the second brain hemisphere probability map, and the first sample segmentation image.

[0084] In a possible implementation, the training of the brain segmentation model based on the first brain hemisphere probability map, the second brain hemisphere probability map, and the first sample segmentation image includes:

[0085] determining, based on the first sample segmented image, at least one point belonging to the first hemisphere region and at least one point belonging to the second hemisphere region;

[0086] Determining, based on the first hemisphere probability map, a probability that the at least one point belonging to the first hemisphere region belongs to the first hemisphere region, and determining, based on the second hemisphere probability map, a probability that the at least one point belonging to the second hemisphere region belongs to the second hemisphere region;

[0087] determining a loss value of the brain segmentation model based on a probability that the at least one point belonging to the first hemisphere region belongs to the first hemisphere region and a probability that the at least one point belonging to the second hemisphere region belongs to the second hemisphere region;

[0088] The brain segmentation model is trained based on the loss value.

[0089] In one possible implementation, the brain segmentation model includes an encoding sub-model and a decoding sub-model, the encoding sub-model includes k encoding networks, and the decoding sub-model includes k decoding networks;

[0090] The calling of the brain segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain a first half brain probability map and a second half brain probability map includes:

[0091] Inputting the sample three-dimensional brain image into a first encoding network, performing feature extraction on the sample three-dimensional brain image based on the first encoding network, and outputting a first encoding feature map;

[0092] Based on the i-th encoding network, feature extraction is performed on the encoding feature map output by the previous network, and the i-th encoding feature map is output, until the k-th encoding network outputs the k-th encoding feature map, and the k-th encoding feature map is input into the first decoding network;

[0093] Reconstructing the k-th encoding feature map based on the first decoding network to obtain a first decoding feature map;

[0094] Based on the i-th decoding network, reconstruct the feature map after fusing the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map, and output the i-th decoding feature map until the k-th decoding network outputs the k-th decoding feature map, wherein the k-th decoding feature map includes the first hemi-brain probability map and the second hemi-brain probability map;

[0095] Wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

[0096] In a possible implementation, the encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks; and the step of extracting features from the encoding feature map output by the previous network based on the i-th encoding network and outputting the i-th encoding feature map includes:

[0097] Based on the i-1th downsampling network, downsampling the encoding feature map output by the i-1th encoding network, outputting the i-1th intermediate encoding feature map, inputting the i-1th intermediate encoding feature map into the i-th encoding network, performing feature extraction on the i-1th intermediate encoding feature map based on the i-th encoding network, and outputting the i-th encoding feature map;

[0098] The method comprises: reconstructing an image of a feature map obtained by fusing a decoding feature map output by a previous network and a (k+1-i)th encoding feature map based on the i-th decoding network, and outputting the i-th decoding feature map.

[0099] Based on the i-1th upsampling network, the decoding feature map output by the i-1th decoding network is upsampled, and the i-1th intermediate decoding feature map is output. The i-1th intermediate decoding feature map is input into the i-th decoding network. Based on the i-th decoding network, the feature map after the fusion of the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map is reconstructed, and the i-th decoding feature map is output.

[0100] In a possible implementation, the first recognition module is configured to identify contour points of the brain region from the three-dimensional brain image; and determine a set of contour points of the brain region as a contour surface of the brain region;

[0101] The second recognition module is used to identify the contour points of the first half of the brain region and the contour points of the second half of the brain region from the three-dimensional brain image; determine the set formed by the contour points of the first half of the brain region as the first half of the brain contour surface, and determine the set formed by the contour points of the second half of the brain region as the second half of the brain contour surface.

[0102] In a possible implementation, the apparatus further includes:

[0103] a state determination module, configured to determine a state corresponding to the brain region based on the three-dimensional brain image, the state being whether the lesion has broken into the ventricle or the lesion has not broken into the ventricle;

[0104] The brain midline correction module is used to correct the three-dimensional brain midline in response to the state that the lesion has not broken into the ventricle.

[0105] In one possible implementation, the brain midline correction module includes:

[0106] a second segmentation unit, configured to segment the three-dimensional brain image to obtain a lesion area;

[0107] A brain midline correction unit is used to determine a target area where the lesion is located based on the lesion area and the three-dimensional brain midline, where the target area is the first hemisphere area or the second hemisphere area; remove the first contour surface in the lesion contour surface of the lesion area that is located in the target area to obtain a remaining second contour surface; determine two intersection lines of the lesion contour surface and the three-dimensional brain midline, remove the first brain midline between the two intersection lines in the three-dimensional brain midline to obtain a remaining second brain midline; fit the second brain midline with the second contour surface to obtain a corrected three-dimensional brain midline.

[0108] In one possible implementation, the state determination module is used to call a classification model to classify the three-dimensional brain image to obtain a classification label, and the classification label is used to describe the state corresponding to the brain area, and the state is whether the lesion has broken into the ventricle or the lesion has not broken into the ventricle.

[0109] In one possible implementation, the training process of the classification model includes:

[0110] Acquiring second sample data, the second sample data including a sample three-dimensional brain image and a sample label, the sample three-dimensional brain image including a brain region, the sample label being used to describe a state corresponding to the brain region, the state being whether the lesion has ruptured into the ventricle or the lesion has not ruptured into the ventricle;

[0111] Calling the classification model to classify the sample three-dimensional brain image to obtain a predicted label;

[0112] The classification model is trained based on the sample labels and the predicted labels.

[0113] In a possible implementation, the second segmentation unit is configured to call a lesion segmentation model to perform image segmentation on the three-dimensional brain image to obtain the lesion area.

[0114] In a possible implementation, the second segmentation unit includes:

[0115] a segmentation subunit, configured to call the lesion segmentation model, perform image segmentation on the three-dimensional brain image, and obtain a lesion probability map, wherein the lesion probability map is used to represent the probability that each point in the three-dimensional brain image belongs to a lesion area;

[0116] The determination subunit is configured to determine points in the lesion probability map whose corresponding probabilities are greater than a second probability threshold as points in the lesion area, and to determine a set of points in the lesion area as the lesion area.

[0117] In a possible implementation, the lesion segmentation model includes an encoding sub-model and a decoding sub-model, the encoding sub-model includes k encoding networks, and the decoding sub-model includes k decoding networks;

[0118] The segmentation subunit is used to input the three-dimensional brain image into the first encoding network, perform feature extraction on the three-dimensional brain image based on the first encoding network, and output the first encoding feature map; based on the i-th encoding network, perform feature extraction on the encoding feature map output by the previous network, output the i-th encoding feature map, until the k-th encoding network outputs the k-th encoding feature map, and input the k-th encoding feature map into the first decoding network; based on the first decoding network, perform image reconstruction on the k-th encoding feature map to obtain the first decoding feature map; based on the i-th decoding network, perform image reconstruction on the feature map after fusion of the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map, output the i-th decoding feature map, until the k-th decoding network outputs the k-th decoding feature map, and determine the k-th decoding feature map as the lesion probability map; wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

[0119] In a possible implementation, the encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks;

[0120] The segmentation subunit is used to downsample the encoding feature map output by the i-1th encoding network based on the i-1th downsampling network, output the i-1th intermediate encoding feature map, input the i-1th intermediate encoding feature map to the i-th encoding network, perform feature extraction on the i-1th intermediate encoding feature map based on the i-th encoding network, and output the i-th encoding feature map; upsample the decoding feature map output by the i-1th decoding network based on the i-1th upsampling network, output the i-1th intermediate decoding feature map, input the i-1th intermediate decoding feature map to the i-th decoding network, perform image reconstruction on the feature map after fusion of the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map based on the i-th decoding network, and output the i-th decoding feature map.

[0121] In one possible implementation, the training process of the lesion segmentation model includes:

[0122] Acquire third sample data, the third sample data including a sample three-dimensional brain image and a second sample segmented image, wherein the second sample segmented image is marked with a lesion area in the sample three-dimensional brain image;

[0123] calling the lesion segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain a lesion probability map, wherein the lesion probability map is used to represent the probability that each point in the sample three-dimensional brain image belongs to a lesion area;

[0124] The lesion segmentation model is trained based on the lesion probability map and the second sample segmentation image.

[0125] In a possible implementation, the training of the lesion segmentation model based on the lesion probability map and the second sample segmentation image includes:

[0126] determining at least one point belonging to a lesion area based on the second sample segmented image;

[0127] Determining, based on the lesion probability map, a probability that the at least one point belonging to the lesion area belongs to the lesion area;

[0128] Determining a loss value of the lesion segmentation model based on a probability that the at least one point belonging to the lesion area belongs to the lesion area;

[0129] Based on the loss value, the lesion segmentation model is trained.

[0130] In a possible implementation, the lesion segmentation model includes an encoding sub-model and a decoding sub-model, the encoding sub-model includes k encoding networks, and the decoding sub-model includes k decoding networks;

[0131] The calling of the lesion segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain a lesion probability map includes:

[0132] Inputting the sample three-dimensional brain image into a first encoding network, performing feature extraction on the sample three-dimensional brain image based on the first encoding network, and outputting a first encoding feature map;

[0133] Based on the i-th encoding network, feature extraction is performed on the encoding feature map output by the previous network, and the i-th encoding feature map is output, until the k-th encoding network outputs the k-th encoding feature map, and the k-th encoding feature map is input into the first decoding network;

[0134] Reconstructing the k-th encoding feature map based on the first decoding network to obtain a first decoding feature map;

[0135] Based on the i-th decoding network, the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map are fused to perform image reconstruction, and the i-th decoding feature map is output, until the k-th decoding network outputs the k-th decoding feature map, and the k-th decoding feature map is determined as the lesion probability map;

[0136] Wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

[0137] In a possible implementation, the encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks; and the step of extracting features from the encoding feature map output by the previous network based on the i-th encoding network and outputting the i-th encoding feature map includes:

[0138] Based on the i-1th downsampling network, downsampling the encoding feature map output by the i-1th encoding network, outputting the i-1th intermediate encoding feature map, inputting the i-1th intermediate encoding feature map into the i-th encoding network, performing feature extraction on the i-1th intermediate encoding feature map based on the i-th encoding network, and outputting the i-th encoding feature map;

[0139] The method comprises: reconstructing an image of a feature map obtained by fusing a decoding feature map output by a previous network and a (k+1-i)th encoding feature map based on the i-th decoding network, and outputting the i-th decoding feature map.

[0140] Based on the i-1th upsampling network, the decoding feature map output by the i-1th decoding network is upsampled, and the i-1th intermediate decoding feature map is output. The i-1th intermediate decoding feature map is input into the i-th decoding network. Based on the i-th decoding network, the feature map after the fusion of the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map is reconstructed, and the i-th decoding feature map is output.

[0141] In one possible implementation, the image acquisition module is used to acquire an original brain image, which is a three-dimensional image; acquire a template brain image, which is bilaterally symmetrical; and perform rigid registration on the original brain image according to the template brain image to obtain the three-dimensional brain image.

[0142] In one possible implementation, the image acquisition module is used to acquire a computed tomography (CT) image of the brain, and determine the window width and window level of the CT image as a reference window width and a reference window level, respectively, to obtain the three-dimensional brain image; wherein the window width represents the range of CT values ​​displayed in the CT image, and the window level represents the middle value of the range of CT values.

[0143] In one possible implementation, the image acquisition module is used to acquire an original brain image, where the original brain image includes multiple scanning layers; sample the multiple scanning layers so that the interval between any two scanning layers reaches a reference interval; and use the sampled original brain image as the three-dimensional brain image.

[0144] In a possible implementation, the image acquisition module is configured to acquire an original brain image, which is a three-dimensional image; and adjust the size of the original brain image to a reference size to obtain the three-dimensional brain image.

[0145] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one computer program, and the computer program is loaded and executed by the processor to implement the operations performed in the brain midline recognition method in any of the above possible implementations.

[0146] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The computer program is loaded and executed by a processor to implement the operations performed in the brain midline identification method in any of the possible implementations described above.

[0147] In another aspect, a computer program product or computer program is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the operations performed in the brain midline identification method in the various optional implementations described above.

[0148] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0149] When identifying the brain midline, the embodiment of the present application converts the problem of identifying the brain midline into the problem of dividing the left and right brain halves, and uses a three-dimensional brain image to divide the left and right brains, thereby obtaining the first half brain contour surface and the second half brain contour surface. The difference between the union of the first half brain contour surface and the second half brain contour surface and the whole brain contour surface is the three-dimensional brain midline. Compared with the brain midline identified by a two-dimensional brain image, this three-dimensional brain midline can provide more information and is more conducive to determining the user's brain state. BRIEF DESCRIPTION OF THE DRAWINGS

[0150] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0151] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0152] Figure 2 This is a flow chart of a brain midline identification method provided in an embodiment of the present application;

[0153] Figure 3 This is a flow chart of a brain midline identification method provided in an embodiment of the present application;

[0154] Figure 4 This is a schematic diagram of a scanning layer provided in an embodiment of the present application;

[0155] Figure 5 This is a flow chart of a brain midline identification method provided in an embodiment of the present application;

[0156] Figure 6 This is a three-dimensional segmentation map of the brain provided in an embodiment of the present application;

[0157] Figure 7 This is a schematic diagram of a scanning layer provided in an embodiment of the present application;

[0158] Figure 8 is a schematic diagram of a three-dimensional brain midline provided in an embodiment of the present application;

[0159] Figure 9 This is a schematic diagram of a scanning layer provided in an embodiment of the present application;

[0160] Figure 10 This is a schematic diagram of a process for identifying a three-dimensional brain midline using a brain segmentation model provided in an embodiment of the present application;

[0161] Figure 11 This is a schematic diagram of a scanning layer provided in an embodiment of the present application;

[0162] Figure 12 This is a schematic diagram of a scanning layer provided in an embodiment of the present application;

[0163] Figure 13 is a schematic diagram of a classification model provided in an embodiment of the present application;

[0164] Figure 14 This is a schematic diagram of a process for identifying a three-dimensional brain midline provided by an embodiment of the present application;

[0165] Figure 15 This is a flow chart of a brain midline identification method provided in an embodiment of the present application;

[0166] Figure 16 This is a block diagram of a brain midline recognition device provided by an embodiment of the present application;

[0167] Figure 17 This is a block diagram of a brain midline recognition device provided by an embodiment of the present application;

[0168] Figure 18 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0169] Figure 19 This is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0170] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0171] As used herein, the terms "first," "second," "third," "fourth," and the like may be used to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are used solely to distinguish one concept from another. For example, without departing from the scope of this application, a first coding network may be referred to as a coding network, and similarly, a second coding network may be referred to as a first coding network.

[0172] As used herein, the terms "at least one," "a plurality," "each," and "any" include one, two, or more, "a plurality" includes two or more, "each" refers to each of the corresponding plurality, and "any" refers to any one of the plurality. For example, the plurality of coding networks includes three coding networks, and "each" refers to each of the three coding networks, and "any" refers to any one of the three coding networks, which can be the first, second, or third.

[0173] Figure 1This is a schematic diagram of an implementation environment provided by an embodiment of the present application. Figure 1 , the implementation environment includes a terminal 101 and a server 102. The terminal 101 and the server 102 are connected via a wireless or wired network. Optionally, the terminal 101 is a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a car terminal, a video camera, a camera or other terminals. Optionally, the server 102 is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0174] Optionally, a target application provided by the server 102 is installed on the terminal 101, and the terminal 101 can implement functions such as data transmission and message interaction through the target application. Optionally, the target application is a target application in the terminal 101 operating system, or a target application provided by a third party. The target application has the function of processing three-dimensional brain images, for example, identifying the three-dimensional brain midline in the three-dimensional brain image. Of course, the target application can also have other functions, which are not limited by this application. Optionally, the target application is an image processing application, a modeling application, etc.

[0175] In an embodiment of the present application, the terminal 101 is used to obtain a three-dimensional brain image and identify a three-dimensional brain midline from the three-dimensional brain image. Alternatively, the terminal 101 uploads the three-dimensional brain image to the server 102, and after the server 102 identifies the three-dimensional brain midline from the three-dimensional brain image, the three-dimensional brain image with the identified three-dimensional brain midline is downloaded to the terminal 101. It should be noted that the embodiment of the present application is only described by taking the implementation environment including the terminal 101 and the server 102 as an example. In other embodiments, the implementation environment only includes the terminal 101 or the server 102. The terminal 101 or the server 102 obtains the three-dimensional brain image and identifies the three-dimensional brain midline therein.

[0176] The brain midline recognition method in this application can be applied in scenarios where the user's brain state is to be judged. For example, after a doctor scans a three-dimensional brain image for a user using a three-dimensional brain image scanning device, the three-dimensional brain image is sent to a device for identifying a three-dimensional brain midline. The device uses the method provided in this application to identify the three-dimensional brain midline in the three-dimensional brain image, helping the doctor to judge the state of the user's brain. Alternatively, the device for scanning the three-dimensional brain image and the device for identifying the three-dimensional brain midline are the same device. After the doctor obtains the user's three-dimensional brain image through the device, he or she directly uses the device to identify the three-dimensional brain midline therein.

[0177] The brain midline recognition method in this application can also be applied in the scenario of surgical navigation. For example, before performing brain surgery on a user, the doctor can use any device to obtain the user's three-dimensional brain midline using the method provided in this application. When performing brain surgery on the user, the three-dimensional brain midline is used as a reference to achieve real-time positioning during the operation, so as to avoid various surgical instruments penetrating the partition membrane between the left and right hemispheres of the brain, etc., thereby improving the safety of the operation.

[0178] The brain midline recognition method in this application can also be used in the scenario of formulating surgical plans. For example, before performing brain surgery on a user, the doctor can use any device to obtain the user's three-dimensional brain midline using the method provided in this application, and then formulate a surgical plan that avoids the three-dimensional brain midline based on the status of the user's brain area.

[0179] The brain midline recognition method of the present application can be applied in the scenario of disease diagnosis. For example, the three-dimensional brain midline obtained according to the method provided by the present application, combined with other information, can determine whether the user is ill and the severity of the disease. Optionally, other information includes a standard three-dimensional brain midline, wherein the standard three-dimensional brain midline is a three-dimensional brain midline without position offset. Combining the three-dimensional brain midline provided by the present application with the standard three-dimensional brain midline, it is possible to determine the offset distance of the user's brain midline, and use this distance as a judgment indicator of brain disease to determine whether the user suffers from diseases such as brain hematoma and brain tumor. It is also possible to determine the degree of compression of brain tissue by diseases such as brain hematoma or brain tumor based on the offset distance, and then determine the severity of the user's brain disease.

[0180] In fact, the brain midline recognition method provided in this application can be applied to any scenario of identifying the brain midline, and the embodiments of this application do not limit this.

[0181] Figure 2 This is a flow chart of a brain midline recognition method provided by an embodiment of the present application. Figure 2 , the embodiment includes:

[0182] 201. A terminal obtains a three-dimensional brain image, where the three-dimensional brain image includes a brain region.

[0183] The brain region in the three-dimensional brain image is a three-dimensional structure. Optionally, the brain region includes multiple points, each of which has three-dimensional coordinates, indicating the position of the point in the brain region. Optionally, the coordinate origin of the three-dimensional coordinate system is at any position in the brain region. For example, the coordinate origin is the position of the nose tip in the brain region, the position of the left eye corner, etc., which is not limited in this embodiment of the present application. The three-dimensional brain image includes a background region and a brain region. The background region is the other region in the three-dimensional brain image except the brain region.

[0184] 202. The terminal identifies the contour surface of the brain region from the three-dimensional brain image.

[0185] The contour surface of the brain region refers to a curved surface formed by multiple contour points of the brain region.

[0186] 203. The terminal performs image segmentation on the three-dimensional brain image to obtain a first hemisphere region and a second hemisphere region.

[0187] Under normal circumstances, the brain has a bilaterally symmetrical structure, and the brain areas include the left and right hemispheres. The left hemisphere refers to the half of the brain located on the right side when viewing a three-dimensional brain image from the front. Correspondingly, the right hemisphere refers to the half of the brain located on the left side when viewing a three-dimensional brain image from the front. Alternatively, the left hemisphere corresponds to the user's actual left brain, and the right hemisphere corresponds to the user's actual right brain. The first hemisphere is the left or right hemisphere, and the second hemisphere is the left or right hemisphere, and the first and second hemispheres are different.

[0188] 204. The terminal identifies a first hemisphere contour surface of the first hemisphere region and a second hemisphere contour surface of the second hemisphere region from the three-dimensional brain image.

[0189] The first hemisphere contour surface refers to a curved surface formed by multiple contour points of the first hemisphere region, and the second hemisphere contour surface refers to a curved surface formed by multiple contour points of the second hemisphere region.

[0190] 205. The terminal determines the difference between the union of the first half brain contour surface and the second half brain contour surface and the contour surface of the brain region as the three-dimensional brain midline of the brain region.

[0191] The difference between the union of the first half brain contour surface and the second half brain contour surface and the contour surface of the brain region is a three-dimensional structure, and this three-dimensional structure separates the first half brain region and the second half brain region. Therefore, this three-dimensional structure is used as the three-dimensional brain midline of the brain region.

[0192] When identifying the brain midline, the embodiment of the present application converts the problem of identifying the brain midline into the problem of dividing the left and right brain halves, and uses a three-dimensional brain image to divide the left and right brains, thereby obtaining the first half brain contour surface and the second half brain contour surface. The difference between the union of the first half brain contour surface and the second half brain contour surface and the whole brain contour surface is the three-dimensional brain midline. Compared with the brain midline identified by a two-dimensional brain image, this three-dimensional brain midline can provide more information and is more conducive to determining the user's brain state.

[0193] Figure 3 This is a flow chart of a brain midline recognition method provided in an embodiment of the present application. This embodiment describes the training process of the brain segmentation model. Figure 3 , the embodiment includes:

[0194] 301. A terminal obtains first sample data, where the first sample data includes a sample three-dimensional brain image and a first sample segmented image. The first sample segmented image is annotated with a first hemisphere region and a second hemisphere region in the sample three-dimensional brain image.

[0195] The brain segmentation model is used to identify the first and second hemispheres in a three-dimensional brain image. This embodiment of the present application trains the brain segmentation model, enabling subsequent rapid identification of the first and second hemispheres in a three-dimensional brain image using the model, thereby improving the efficiency of brain midline recognition.

[0196] Wherein, the sample three-dimensional brain image is a three-dimensional brain image used to train the brain segmentation model, and the number of sample three-dimensional brain images is arbitrary, and the embodiments of the present application do not impose any restrictions on this. The first sample segmented image is marked with a first hemibrain region and a second hemibrain region. Optionally, the brain region is marked in the first sample segmented image, and multiple segmentation points are marked in the middle of the first hemibrain region and the second hemibrain region, and then the multiple segmentation points are connected to obtain a segmentation plane, then the area on the first side of the segmentation plane is the first hemibrain region, and the area on the second side of the segmentation plane is the second hemibrain region. Figure 4 A schematic diagram of a scanning layer in a three-dimensional brain image. A scanning layer is a frame of a two-dimensional brain image. Figure 4 The scan layer includes 11 segmentation points, namely segmentation points F-1 to F-11. The line connecting these segmentation points is the segmentation line corresponding to the scan layer. The three-dimensional structure formed by the segmentation lines corresponding to all scan layers in the three-dimensional brain image is the three-dimensional brain midline.

[0197] Optionally, the terminal obtains the first sample data by: obtaining multiple frames of continuous two-dimensional brain images, reconstructing the multiple frames of continuous two-dimensional brain images into a three-dimensional brain image, and determining the three-dimensional brain image as a sample three-dimensional brain image. Then, the first half brain region and the second half brain region in the sample three-dimensional brain image are marked to obtain a first sample segmented image. The two-dimensional brain image is a CT (Computed Tomography) image of the brain, an MRI (Magnetic Resonance Imaging) image of the brain, or other types of brain scan images.

[0198] In one possible implementation, the process of the terminal acquiring a sample three-dimensional brain image further includes: the terminal acquiring a template brain image, the template brain image being bilaterally symmetrical, the terminal rigidly registering the reconstructed three-dimensional brain image according to the template brain image, and determining the rigidly registered three-dimensional brain image as the sample three-dimensional brain image. Optionally, the template brain image is bilaterally symmetrical and the corresponding three-dimensional brain midline is not offset. The implementation method of the terminal rigidly registering the reconstructed three-dimensional brain image according to the template brain image and determining the rigidly registered three-dimensional brain image as the sample three-dimensional brain image is as follows: the terminal rigidly registers the reconstructed three-dimensional brain image to the template brain image, and determines the registered three-dimensional brain image as the sample three-dimensional brain image.

[0199] Optionally, the terminal rigidly registers the reconstructed three-dimensional brain image with the template brain image by using an image registration tool to rigidly register the original brain image with the template brain image. Optionally, the image registration tool is Elastix (an image registration tool).

[0200] In an embodiment of the present application, the brain segmentation model is trained using sample three-dimensional brain images that have undergone rigid registration. When there are multiple sample three-dimensional brain images, the brain segmentation model will not be disturbed by the different angles of the multiple sample three-dimensional brain images, thereby improving the training efficiency of the brain segmentation model and improving the segmentation accuracy of the brain segmentation model.

[0201] Optionally, the method for obtaining a template brain image includes: obtaining multiple frames of continuous two-dimensional brain images corresponding to a normal brain, the scanning area corresponding to the multiple frames of two-dimensional brain images covers the complete head structure, for example, the scanning area covers from the top of the head to the chin, and the multiple frames of two-dimensional brain images are symmetrical on the left and right, and performing three-dimensional reconstruction on the multiple frames of continuous two-dimensional brain images to obtain a template brain image. The normal brain is a brain judged to be normal by a doctor. The two-dimensional brain image is a brain CT image, a brain MRI image or other types of brain scan images, which is not limited in the embodiments of the present application.

[0202] In the embodiment of the present application, since the template brain image includes a complete brain structure and is bilaterally symmetrical, the reconstructed three-dimensional brain image can be registered when it corresponds to a complete brain structure or a partial brain structure.

[0203] In one possible implementation, when the two-dimensional brain image acquired by the terminal is a CT image of the brain, before reconstructing the two-dimensional brain image into a three-dimensional brain image, the terminal first determines the window width and window level of the brain CT image as a reference window width and a reference window level, respectively, wherein the window width represents the range of CT values ​​displayed in the CT image, and the window level represents the middle value of the range of CT values. The reference window width and reference window level are set as needed. For example, the reference window width and reference window level are the window width and window level that make the brain CT image display the clearest. In an embodiment of the present application, by determining the window width and window level of the brain CT image as the reference window width and reference window level, respectively, the clarity of the CT image can be adjusted to achieve optimal display of brain tissue, hematoma, tumor, and other tissues. Then, by using the sample three-dimensional brain image reconstructed from the CT image to train the brain segmentation model, the training difficulty of the brain segmentation model can be reduced and the accuracy of the brain segmentation model in performing brain segmentation can be improved.

[0204] Optionally, after adjusting the window width and window position of the brain CT image, the terminal also normalizes the pixel values ​​of the CT image, which can reduce the difficulty of learning the brain segmentation model.

[0205] In one possible implementation, a terminal acquires a sample three-dimensional brain image, including: acquiring an original brain image, the original brain image comprising multiple scan layers; sampling the multiple scan layers so that the interval between any two scan layers reaches a reference interval; and using the sampled original brain image as the sample three-dimensional brain image. A scan layer is a frame of a two-dimensional brain image, and the interval between any two scan layers is called a slice thickness, which represents the actual interval between brain layers corresponding to the two scan layers. Optionally, sampling the multiple scan layers so that the interval between any two scan layers reaches the reference interval includes: if the interval between any two scan layers in the original brain image is less than the reference interval, extracting a portion of the scan layers from the multiple scan layers so that the interval between any two scan layers reaches the reference interval. If the interval between any two scan layers in the original brain image is greater than the reference interval, performing linear interpolation between the two scan layers so that the interval between any two scan layers reaches the reference interval. The reference interval is any interval, such as 1 mm or 2 mm, and is not limited in this embodiment of the present application. Optionally, the terminal acquires the original brain image in a manner including: the terminal acquires multiple frames of continuous two-dimensional brain images, reconstructs the multiple frames of continuous two-dimensional brain images into a three-dimensional brain image, and determines the three-dimensional brain image as the original brain image.

[0206] In an embodiment of the present application, by sampling, the interval between any two scanning layers in the sample three-dimensional brain image is made into a reference interval, and then the brain segmentation model is trained using the sample three-dimensional brain image. Even if there are multiple sample three-dimensional brain images, the brain segmentation model will not be disturbed by the different intervals between the scanning layers in the multiple sample three-dimensional brain images, which can improve the training efficiency of the brain segmentation model and the segmentation accuracy of the brain segmentation model.

[0207] In one possible implementation, a terminal acquires a sample three-dimensional brain image, including: acquiring an original brain image, where the original brain image is a three-dimensional image; and resizing the original brain image to a reference size to obtain the sample three-dimensional brain image. Optionally, the terminal resizing the original brain image to the reference size includes: resizing the original brain image to the reference size by cropping or adding pixels to the original brain image, wherein the pixel values ​​of the added pixels are zero.

[0208] In an embodiment of the present application, the size of the sample three-dimensional brain image is adjusted to a reference size, and the brain segmentation model is trained using the sample three-dimensional brain image. Even if there are multiple sample three-dimensional brain images, the brain segmentation model will not be disturbed by the different sizes of the multiple sample three-dimensional brain images, which can improve the training efficiency of the brain segmentation model and the segmentation accuracy of the brain segmentation model.

[0209] 302. The terminal calls the brain segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain a first hemisphere probability map and a second hemisphere probability map.

[0210] The first half brain probability map is used to represent the probability that each point in the sample three-dimensional brain image belongs to the first half brain area, and the second half brain probability map is used to represent the probability that each point in the sample three-dimensional brain image belongs to the second half brain area.

[0211] In one possible implementation, the brain segmentation model includes an encoding sub-model and a decoding sub-model, the encoding sub-model includes k encoding networks, and the decoding sub-model includes k decoding networks, where k is a positive integer greater than 1. Accordingly, the terminal calls the brain segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain a first hemisphere probability map and a second hemisphere probability map, including the following steps (1)-(4):

[0212] (1) The terminal inputs the sample three-dimensional brain image into the first encoding network, extracts features of the sample three-dimensional brain image based on the first encoding network, and outputs the first encoding feature map.

[0213] (2) Based on the i-th encoding network, the terminal extracts features from the encoding feature map output by the previous network and outputs the i-th encoding feature map until the k-th encoding network outputs the k-th encoding feature map, and inputs the k-th encoding feature map into the first decoding network.

[0214] Wherein, i is any positive integer greater than 1 and not greater than k. Optionally, the previous network of the i-th coding network is the i-1-th coding network.

[0215] (3) The terminal reconstructs the kth encoded feature map based on the first decoding network to obtain the first decoded feature map.

[0216] (4) Based on the i-th decoding network, the terminal reconstructs the image of the feature map after fusing the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map, and outputs the i-th decoding feature map until the k-th decoding network outputs the k-th decoding feature map. The k-th decoding feature map includes the first hemisphere probability map and the second hemisphere probability map.

[0217] Optionally, the previous network of the i-th decoding network is the i-1-th decoding network. Optionally, the implementation method of fusing the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map based on the i-th decoding network is as follows: based on the i-th decoding network, the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map are connected (concat) to obtain a fused feature map. Alternatively, based on the i-th decoding network, the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map are added (add) to obtain a fused feature map, and the embodiment of the present application does not impose any restrictions on this. Optionally, before the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map are connected based on the i-th decoding network, the (k+1-i)-th encoding feature map is cut to the same size as the decoding feature map output by the previous network. Figure 1 Then make the connection operation.

[0218] The encoding network has an arbitrary structure. For example, the encoding network includes two 3*3*3 convolution layers and an activation layer. Optionally, the activation layer uses a rectified linear unit (ReLU). Furthermore, the structure of each encoding network is the same or different. The decoding network has an arbitrary structure. For example, the decoding network includes two 3*3*3 convolution layers and an activation layer, or the decoding network includes one 1*1*1 convolution layer. Furthermore, the structure of each decoding network is the same or different.

[0219] In one possible implementation, the encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks. Accordingly, the terminal extracts features from the encoding feature map output by the previous network based on the i-th encoding network and outputs the i-th encoding feature map, including: the terminal downsamples the encoding feature map output by the i-1-th encoding network based on the i-1-th downsampling network, outputs the i-1-th intermediate encoding feature map, inputs the i-1-th intermediate encoding feature map to the i-th encoding network, extracts features from the i-1-th intermediate encoding feature map based on the i-th encoding network, and outputs the i-th encoding feature map. Downsampling is used to reduce the dimension of the encoding feature map.

[0220] The terminal, based on the i-th decoding network, reconstructs an image of a feature map obtained by fusing the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map, and outputs the i-th decoding feature map, including: the terminal upsampling the decoding feature map output by the i-1-th decoding network based on the i-1-th upsampling network, outputting the i-1-th intermediate decoding feature map, inputting the i-1-th intermediate decoding feature map into the i-th decoding network, and reconstructing an image of a feature map obtained by fusing the i-1-th intermediate decoding feature map and the (k+1-i)-th encoding feature map based on the i-th decoding network, and outputting the i-th decoding feature map. The upsampling is used to increase the dimension of the encoding feature map.

[0221] Among them, based on the i-th decoding network, the implementation method of fusing the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map is the same as the implementation method of fusing the decoding feature map output by the previous network and the (k+1-i)th encoding feature map, which will not be repeated here.

[0222] The upsampling network can have any structure. For example, the upsampling network includes a 2*2*2 convolutional layer. Furthermore, the structures of each upsampling network can be the same or different. The downsampling network can have any structure. For example, the downsampling network includes a 2*2*2 pooling layer. Furthermore, the structures of each downsampling network can be the same or different.

[0223] In an embodiment of the present application, when performing image segmentation using a brain segmentation model to obtain a probability map of the first and second hemispheres, downsampling is performed by the downsampling network in the encoding sub-model to obtain low-resolution intermediate encoding features, thereby providing an identification basis for the first and second hemispheres. Upsampling is performed by the upsampling network in the decoding sub-model to obtain high-resolution intermediate decoding features, thereby providing a positioning basis for the precise segmentation of the first and second hemispheres. This ensures the accuracy of the probability map of the first and second hemispheres determined by the brain segmentation model. In addition, when performing image reconstruction, the decoding network following the first decoding network fills the underlying information in the decoding feature map by fusing the encoding feature map output by the encoding network with the decoding feature map output by the decoding network, thereby improving the segmentation accuracy of the brain segmentation model.

[0224] 303. The terminal trains a brain segmentation model based on the first brain hemisphere probability map, the second brain hemisphere probability map, and the first sample segmentation image.

[0225] In one possible implementation, the terminal trains a brain segmentation model based on the first-hemisphere probability map, the second-hemisphere probability map, and the first sample segmentation image, including: the terminal determines, based on the first sample segmentation image, at least one point belonging to the first-hemisphere region and at least one point belonging to the second-hemisphere region; based on the first-hemisphere probability map, determines the probability that at least one point belonging to the first-hemisphere region belongs to the first-hemisphere region, and based on the second-hemisphere probability map, determines the probability that at least one point belonging to the second-hemisphere region belongs to the second-hemisphere region; then, the terminal determines a loss value of the brain segmentation model based on the probability that at least one point belonging to the first-hemisphere region belongs to the first-hemisphere region and the probability that at least one point belonging to the second-hemisphere region belongs to the second-hemisphere region; and the brain segmentation model is trained based on the loss value.

[0226] Optionally, the terminal determines the loss value of the brain segmentation model based on the probability that at least one point belonging to the first hemisphere region belongs to the first hemisphere region and the probability that at least one point belonging to the second hemisphere region belongs to the second hemisphere region by: determining the first hemisphere loss value based on the probability that at least one point in the first hemisphere region belongs to the first hemisphere region, determining the second hemisphere loss value based on the probability that at least one point in the second hemisphere region belongs to the second hemisphere region, and determining the sum of the first hemisphere loss value and the second hemisphere loss value as the loss value of the brain segmentation model. Optionally, the loss value of the brain segmentation model is determined using the following formulas (1) and (2).

[0227]

[0228] L total =L L +λ*L R (2)

[0229] Among them, L dice Indicates the first half brain loss value or the second half brain loss value. dice In the case of representing the loss value of the first hemisphere, X represents the probability that at least one point in the first hemisphere region determined by the brain segmentation model belongs to the first hemisphere region, and Y represents the probability that at least one point in the first hemisphere region actually belongs to the first hemisphere region, that is, 1. dice In the case of L representing the loss value of the second hemisphere, X represents the probability that at least one point in the second hemisphere region determined by the brain segmentation model belongs to the second hemisphere region, and Y represents the probability that at least one point in the second hemisphere region actually belongs to the second hemisphere region, that is, 1. total Represents the loss value of the brain segmentation model, L L Indicates the first half brain loss value, L R represents the loss value of the second hemisphere, and λ represents the weight of the loss value of the second hemisphere.

[0230] Optionally, the terminal trains the brain segmentation model based on the loss value in the following manner: the terminal adjusts the parameters of the brain segmentation model so that the loss value obtained based on the adjusted brain segmentation model becomes smaller.

[0231] In an embodiment of the present application, at least one point that actually belongs to the first brain hemisphere and at least one point that belongs to the second brain hemisphere are determined by the first segmented image. The probabilities that the at least one point belonging to the first brain hemisphere belongs to the first brain hemisphere and the probability that the at least one point belonging to the second brain hemisphere belongs to the second brain hemisphere, determined by the brain segmentation model, can reflect the segmentation accuracy of the brain segmentation model. Therefore, the loss value of the brain segmentation model is determined by these two probabilities, and training the brain segmentation model according to the loss value can continuously improve the segmentation accuracy of the model.

[0232] In an embodiment of the present application, the brain segmentation model is trained using sample three-dimensional brain images that have undergone rigid registration. When there are multiple sample three-dimensional brain images, the brain segmentation model will not be disturbed by the different angles of the multiple sample three-dimensional brain images, thereby improving the training efficiency of the brain segmentation model and improving the segmentation accuracy of the brain segmentation model.

[0233] In an embodiment of the present application, by determining the window width and window position of the brain CT image as the reference window width and reference window position respectively, the clarity of the CT image can be adjusted. Then, the brain segmentation model is trained by using the sample three-dimensional brain image reconstructed from the CT image, which can reduce the training difficulty of the brain segmentation model and improve the accuracy of brain segmentation performed by the brain segmentation model.

[0234] In an embodiment of the present application, by sampling, the interval between any two scanning layers in the sample three-dimensional brain image is made into a reference interval, and then the brain segmentation model is trained using the sample three-dimensional brain image. Even if there are multiple sample three-dimensional brain images, the brain segmentation model will not be disturbed by the different intervals between the scanning layers in the multiple sample three-dimensional brain images, which can improve the training efficiency of the brain segmentation model and the segmentation accuracy of the brain segmentation model.

[0235] In an embodiment of the present application, the size of the sample three-dimensional brain image is adjusted to a reference size, and the brain segmentation model is trained using the sample three-dimensional brain image. Even if there are multiple sample three-dimensional brain images, the brain segmentation model will not be disturbed by the different sizes of the multiple sample three-dimensional brain images, which can improve the training efficiency of the brain segmentation model and the segmentation accuracy of the brain segmentation model.

[0236] In an embodiment of the present application, when performing image segmentation using a brain segmentation model to obtain a probability map of the first and second hemispheres, downsampling is performed by the downsampling network in the encoding sub-model to obtain low-resolution intermediate encoding features, thereby providing an identification basis for the first and second hemispheres. Upsampling is performed by the upsampling network in the decoding sub-model to obtain high-resolution intermediate decoding features, thereby providing a positioning basis for the precise segmentation of the first and second hemispheres. This ensures the accuracy of the probability map of the first and second hemispheres determined by the brain segmentation model. In addition, when performing image reconstruction, the decoding network following the first decoding network fills the underlying information in the decoding feature map by fusing the encoding feature map output by the encoding network with the decoding feature map output by the decoding network, thereby improving the segmentation accuracy of the brain segmentation model.

[0237] In an embodiment of the present application, at least one point that actually belongs to the first brain hemisphere and at least one point that belongs to the second brain hemisphere are determined by the first segmented image. The probabilities that the at least one point belonging to the first brain hemisphere belongs to the first brain hemisphere and the probability that the at least one point belonging to the second brain hemisphere belongs to the second brain hemisphere, determined by the brain segmentation model, can reflect the segmentation accuracy of the brain segmentation model. Therefore, the loss value of the brain segmentation model is determined by these two probabilities, and training the brain segmentation model according to the loss value can continuously improve the segmentation accuracy of the model.

[0238] Figure 5 This is a flow chart of a method for identifying the brain midline provided by an embodiment of the present application. This embodiment describes the process of identifying the brain midline. Figure 5 , the embodiment includes:

[0239] 501. A terminal obtains a three-dimensional brain image, where the three-dimensional brain image includes a brain region.

[0240] In one possible implementation, the terminal obtains a three-dimensional brain image, including: the terminal obtains an original brain image, which is a three-dimensional image; obtains a template brain image, which is bilaterally symmetrical; the terminal rigidly aligns the original brain image with the template brain image to obtain a three-dimensional brain image. The implementation method of this step is the same as the registration method when obtaining a sample three-dimensional brain image, and will not be repeated here. In the embodiment of the present application, considering that the sample three-dimensional brain image used in training the brain segmentation model is bilaterally symmetrical, the template brain image is used to rigidly align the original brain image so that the obtained three-dimensional brain image is bilaterally symmetrical. When the brain segmentation model is subsequently used to segment the three-dimensional brain image, the segmentation difficulty of the brain segmentation model can be reduced, thereby ensuring the accuracy of the image segmentation performed by the brain segmentation model.

[0241] In one possible implementation, the terminal acquires a three-dimensional brain image, including: the terminal acquires a CT image of the brain, and determines the window width and window level of the CT image as a reference window width and reference window level, respectively, to obtain the three-dimensional brain image. The implementation method of this step is similar to the method of determining the window width and window level when acquiring a sample three-dimensional brain image, and will not be repeated here. In the embodiment of the present application, by adjusting the window width and window level of the CT image, the clarity of the CT image can be adjusted, thereby reducing the difficulty of image segmentation by the brain segmentation model and ensuring the accuracy of image segmentation performed by the brain segmentation model.

[0242] Optionally, after adjusting the window width and window position of the brain CT image, the terminal also normalizes the pixel values ​​of the CT image, which can reduce the difficulty of learning the brain segmentation model.

[0243] In one possible implementation, the terminal acquires a three-dimensional brain image, including: the terminal acquires an original brain image, the original brain image includes multiple scanning layers; the multiple scanning layers are sampled so that the interval between any two scanning layers reaches the reference interval; the sampled original brain image is used as a three-dimensional brain image. The implementation method of this step is the same as the sampling method when acquiring a sample three-dimensional brain image, and will not be repeated here. In the embodiment of the present application, considering that when the brain segmentation model is trained, the interval between any two scanning layers in the sample three-dimensional brain image used is the reference interval, therefore, the original brain image is sampled so that the interval between any two scanning layers in the acquired three-dimensional brain image is also the reference interval. Then, when the brain segmentation model is used to perform image segmentation on the three-dimensional brain image, the segmentation difficulty of the brain segmentation model can be reduced, thereby ensuring the accuracy of the image segmentation performed by the brain segmentation model.

[0244] In one possible implementation, the terminal obtains a three-dimensional brain image, including: the terminal obtains an original brain image, which is a three-dimensional image; and adjusts the size of the original brain image to a reference size to obtain a three-dimensional brain image. The implementation method of this step is the same as the method of resizing when obtaining a sample three-dimensional brain image, and will not be repeated here. In an embodiment of the present application, considering that the size of the sample three-dimensional brain image used in training the brain segmentation model is a reference size, the original brain image is sampled so that the size of the obtained three-dimensional brain image is also a reference size. When the brain segmentation model is subsequently used to segment the three-dimensional brain image, the segmentation difficulty of the brain segmentation model can be reduced, thereby ensuring the accuracy of the image segmentation performed by the brain segmentation model.

[0245] 502. The terminal identifies the contour surface of the brain region from the three-dimensional brain image.

[0246] In one possible implementation, the terminal identifies the contour surface of the brain region from the three-dimensional brain image, including: the terminal identifies the contour points of the brain region from the three-dimensional brain image; and determines the set of contour points of the brain region as the contour surface of the brain region.

[0247] Optionally, the terminal identifies the contour points of a brain region from a three-dimensional brain image by calculating an image gradient of the brain region in the three-dimensional brain image and determining the contour points of the brain region based on the calculated image gradient. The image gradient represents the rate of change of pixel values ​​between adjacent points, and the contour points of a brain region are generally points with a greater rate of change of pixel values ​​than adjacent pixel points. Therefore, by calculating the image gradient, the terminal can determine the contour points of the brain region.

[0248] 503. The terminal calls the brain segmentation model to perform image segmentation on the three-dimensional brain image to obtain the first hemisphere region and the second hemisphere region.

[0249] In one possible implementation, the terminal calls a brain segmentation model to perform image segmentation on a three-dimensional brain image to obtain a first hemisphere region and a second hemisphere region, including the following steps (A)-(C):

[0250] (A) The terminal calls the brain segmentation model to perform image segmentation on the three-dimensional brain image and obtain the probability map of the first half of the brain and the probability map of the second half of the brain.

[0251] The first half brain probability map is used to represent the probability that each point in the three-dimensional brain image belongs to the first half brain region, and the second half brain probability map is used to represent the probability that each point in the three-dimensional brain image belongs to the second half brain region.

[0252] In one possible implementation, the brain segmentation model includes an encoding sub-model and a decoding sub-model, the encoding sub-model includes k encoding networks, and the decoding sub-model includes k decoding networks. Accordingly, the terminal calls the brain segmentation model to perform image segmentation on the three-dimensional brain image to obtain the first half-brain probability map and the second half-brain probability map, including: the terminal inputs the three-dimensional brain image into the first encoding network, extracts features of the three-dimensional brain image based on the first encoding network, and outputs the first encoding feature map; the terminal extracts features from the encoding feature map output by the previous network based on the i-th encoding network, outputs the i-th encoding feature map, until the k-th encoding network outputs the k-th encoding feature map, and inputs the k-th encoding feature map into the first decoding network. In the network, the terminal reconstructs the image of the k-th encoding feature map based on the first decoding network to obtain the first decoding feature map; the terminal reconstructs the image of the feature map after fusing the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map based on the i-th decoding network, and outputs the i-th decoding feature map, until the k-th decoding network outputs the k-th decoding feature map, and the k-th decoding feature map includes the first half-brain probability map and the second half-brain probability map; wherein k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

[0253] In a possible implementation, the encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks. Accordingly, the terminal extracts features from the encoding feature map output by the previous network based on the i-th encoding network and outputs the i-th encoding feature map, including: the terminal downsamples the encoding feature map output by the i-1-th encoding network based on the i-1-th downsampling network, outputs the i-1-th intermediate encoding feature map, inputs the i-1-th intermediate encoding feature map to the i-th encoding network, extracts features from the i-1-th intermediate encoding feature map based on the i-th encoding network, and outputs the i-th encoding feature map; the terminal extracts features from the decoding feature map output by the previous network based on the i-th decoding network. The terminal upsamples the decoding feature map output by the i-1th decoding network based on the i-1th upsampling network, outputs the i-1th intermediate decoding feature map, inputs the i-1th intermediate decoding feature map to the i-th decoding network, and reconstructs the feature map after fusing the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map based on the i-th decoding network, and outputs the i-th decoding feature map.

[0254] It should be noted that the terminal calls the brain segmentation model to perform image segmentation on the three-dimensional brain image to obtain the probability map of the first half of the brain and the probability map of the second half of the brain. The implementation method is similar to the terminal calling the brain segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain the probability map of the first half of the brain and the probability map of the second half of the brain, which will not be repeated here.

[0255] In an embodiment of the present application, when performing image segmentation using a brain segmentation model to obtain a probability map of the first and second hemispheres, downsampling is performed by the downsampling network in the encoding sub-model to obtain low-resolution intermediate encoding features, thereby providing an identification basis for the first and second hemispheres. Upsampling is performed by the upsampling network in the decoding sub-model to obtain high-resolution intermediate decoding features, thereby providing a positioning basis for the precise segmentation of the first and second hemispheres. This ensures the accuracy of the probability map of the first and second hemispheres determined by the brain segmentation model. In addition, when performing image reconstruction, the decoding network following the first decoding network fills the underlying information in the decoding feature map by fusing the encoding feature map output by the encoding network with the decoding feature map output by the decoding network, thereby improving the segmentation accuracy of the brain segmentation model.

[0256] (B) The terminal determines, in the first hemisphere probability map, points whose corresponding probabilities are greater than a first probability threshold as points in the first hemisphere region, and determines the set of points in the first hemisphere region as the first hemisphere region.

[0257] (C) The terminal determines, in the second hemisphere probability map, points whose corresponding probabilities are greater than the first probability threshold as points in the second hemisphere region, and determines the set of points in the second hemisphere region as the second hemisphere region.

[0258] The first probability threshold can be set to any value as needed, and the present embodiment does not limit this. Optionally, the points in the first hemisphere and the second hemisphere have three-dimensional coordinates, indicating the locations of the points in the brain regions.

[0259] In an embodiment of the present application, a three-dimensional brain image is segmented using a brain segmentation model to obtain a first-half brain probability map and a second-half brain probability map, and then the first-half brain region and the second-half brain region are determined by threshold segmentation. The determination of the first-half brain region and the second-half brain region is efficient and accurate.

[0260] It should be noted that in step 502 above, before determining the contour surface of the brain region, the terminal must first segment the three-dimensional brain image to obtain the brain region. The terminal performs segmentation on the three-dimensional brain image to obtain the brain region in a manner similar to the method for segmenting the three-dimensional brain image to obtain the first and second hemisphere regions. Alternatively, the terminal may first perform step 503 and then step 502, and during step 502, the union of the first and second hemisphere regions identified in step 503 is determined as the brain region.

[0261] Figure 6 The 3D segmentation map of the brain includes the first hemisphere and the second hemisphere. The 3D segmentation map can be obtained by performing image segmentation on the 3D brain image using the brain segmentation model. Figure 7 A schematic diagram of a scanning layer in the 3D segmentation map. Figure 7 The brain area in this scan layer is divided into the first hemisphere area and the second hemisphere area.

[0262] 504. The terminal identifies a first hemisphere contour surface of the first hemisphere region and a second hemisphere contour surface of the second hemisphere region from the three-dimensional brain image.

[0263] In one possible implementation, a terminal identifies a first hemisphere contour surface of a first hemisphere region and a second hemisphere contour surface of a second hemisphere region from a three-dimensional brain image, including: the terminal identifying contour points of the first hemisphere region and contour points of the second hemisphere region from the three-dimensional brain image; and the terminal determining a set of contour points of the first hemisphere region as the first hemisphere contour surface, and determining a set of contour points of the second hemisphere region as the second hemisphere contour surface. The terminal identifying the contour points of the first hemisphere region and the contour points of the second hemisphere region from the three-dimensional brain image is implemented similarly to identifying the contour points of brain regions from the three-dimensional brain image and is not further described here.

[0264] 505. The terminal determines the difference between the union of the first half brain contour surface and the second half brain contour surface and the contour surface of the brain region as the three-dimensional brain midline of the brain region.

[0265] In one possible implementation, this step includes: the terminal determining a difference between the union of the first set and the second set and a third set, and determining the difference as the three-dimensional brain midline. The first set is a set consisting of contour points of the first hemisphere region, the second set is a set consisting of contour points of the second hemisphere region, and the third set is a set consisting of contour points of the brain region.

[0266] Figure 8 is a schematic diagram of the three-dimensional brain midline, refer to Figure 8 , the three-dimensional brain midline is a three-dimensional curved surface structure. Figure 9 A schematic diagram of a scan layer in a 3D brain image, see Figure 9 From this scan layer, we can see a line between the first and second hemispheres. This line is what constitutes Figure 8 One of the three-dimensional brain midlines in .

[0267] Figure 10 Schematic diagram of the process of identifying the 3D brain midline using the brain segmentation model. Figure 10 The brain segmentation model is a U-shaped structure. The first side of the U-shaped structure is the encoding sub-model, and the second side of the U-shaped structure is the decoding sub-model. The U-shaped structure includes five encoding networks (not shown in the figure) and five decoding networks (not shown in the figure) from top to bottom. The first side is the encoding network, and the second side is the decoding network. Figure 10 The arrows and dotted boxes in the figure represent operations, and the cubes of different widths and heights represent feature maps. Among them, the cubes on the first side of the U-shaped structure represent encoding feature maps, and the cubes on the second side of the U-shaped structure represent decoding feature maps. The three-dimensional brain image is input into the brain segmentation model, and the three-dimensional brain image is subjected to feature extraction (including convolution operations and nonlinear operations) based on the first encoding network to obtain the first encoding feature map. The first encoding feature map is downsampled based on the first downsampling network (not shown in the figure) to obtain an intermediate encoding feature map, which is input into the second encoding network, and so on, until the fifth encoding network inputs the obtained encoding feature map into the first decoding network. After the first decoding network reconstructs the encoded feature map (including convolution and nonlinear operations), the obtained decoded feature map is input into the first upsampling network (not shown in the figure). The first upsampling network upsamples the decoded feature map to obtain an intermediate decoded feature map, which is input into the second decoding network. The second decoding network copies the encoded feature map output by the fourth encoding network and cuts the encoded feature map to the same position as the intermediate decoded feature map. Figure 1The size of the sample is then connected to the sheared coding feature map and the intermediate decoding feature map, and the connected feature map is reconstructed to obtain a second decoding feature map. The second decoding feature map is input to a second upsampling network (not shown in the figure), and so on, until the fifth decoding network outputs the fifth decoding feature map, which includes the first half brain probability map and the second half brain probability map. According to the reference threshold, the first half brain probability map and the second half brain probability map are threshold segmented respectively to obtain the left and right half brain segmentation results, i.e., the first half brain region and the second half brain region. The image gradient is calculated for the first half brain region, the second half brain region and the brain region respectively, and the contour surface of the first half brain, the contour surface of the second half brain and the contour surface of the brain region are determined. Then the difference between the union of the first half brain contour surface and the second half brain contour surface and the contour surface of the brain region is determined, and the difference is determined as the three-dimensional brain midline.

[0268] 506. The terminal determines a state corresponding to the brain region based on the three-dimensional brain image, which state is whether the lesion has broken into the ventricle or the lesion has not broken into the ventricle.

[0269] Lesions that have ruptured the ventricles refer to lesions that have ruptured the ventricles and entered the interior of the ventricles. In this case, the lesion exerts less pressure on the 3D brain midline and the 3D brain midline shifts less. Lesions that have not ruptured the ventricles refer to lesions that have not entered the ventricles. In this case, the lesion exerts greater pressure on the 3D brain midline and the 3D brain midline shifts more significantly.

[0270] In one possible implementation, the terminal determines the state corresponding to the brain region based on the three-dimensional brain image, including: the terminal calls a classification model to classify the three-dimensional brain image to obtain a classification label, and the classification label is used to describe the state corresponding to the brain region, where the state is either the lesion has broken into the ventricle or the lesion has not broken into the ventricle. Among them, the function of the classification model is to classify the state corresponding to the brain region in the three-dimensional brain image. In the embodiment of the present application, the classification model is directly called to classify the three-dimensional brain image, thereby determining the state corresponding to the brain region, and the efficiency of determining the state is high.

[0271] In one possible implementation, the training process of the classification model includes: the terminal obtains second sample data, the second sample data includes a sample three-dimensional brain image and a sample label, the sample three-dimensional brain image includes a brain area, and the sample label is used to describe the state corresponding to the brain area, the state is that the lesion has broken into the ventricle or the lesion has not broken into the ventricle; the terminal calls the classification model, classifies the sample three-dimensional brain image, and obtains a predicted label; the terminal trains the classification model based on the sample label and the predicted label.

[0272] Optionally, the sample label is 0 or 1, where 0 indicates that the state corresponding to the brain region is that the lesion has not broken into the ventricle, and 1 indicates that the state corresponding to the brain region is that the lesion has broken into the ventricle. The prediction label is any value in the range of 0-1, which indicates the probability that the state corresponding to the brain region predicted by the classification model is that the lesion has broken into the ventricle or the lesion has not broken into the ventricle. For example, if the prediction label of a sample three-dimensional brain image predicted by the classification model is 1, it means that the classification model predicts that the state corresponding to the brain region in the sample three-dimensional brain image is that the lesion has broken into the ventricle. Optionally, the terminal trains the classification model based on the sample label and the prediction label in the following manner: the terminal determines the loss value of the classification model based on the sample label and the prediction label, and trains the classification model based on the loss value. Optionally, the terminal determines the loss value of the classification model by the following formula (3):

[0273]

[0274] Among them, L BCE Represents the loss value of the classification model, M represents the number of second sample data, y n represents the sample label of the nth sample 3D brain image, x n The predicted label for the nth sample 3D brain image.

[0275] Optionally, the classification model is an arbitrary structure. For example, the classification model includes a convolutional neural network ResNet-50 (a convolutional neural network) and a fully connected layer, wherein ResNet-50 is used to extract image features and the fully connected layer is used to perform classification based on the extracted image features. Figure 13 This is a schematic diagram of the classification model. Figure 13 The classification model includes ResNet-50 and a fully connected layer. When performing image classification, a 3D brain image is input into ResNet-50, and the fully connected layer outputs the corresponding state of the brain region.

[0276] Optionally, the terminal determines the state corresponding to the brain area by other means, which is not limited in the embodiments of the present application.

[0277] 507. In response to the status that the lesion has not broken into the ventricle, the terminal corrects the three-dimensional brain midline.

[0278] Optionally, the lesion is a hematoma, a tumor or other lesions, which is not limited in the embodiments of the present application.

[0279] In one possible implementation, in response to a status that the lesion has not broken into the ventricle, the terminal corrects the three-dimensional brain midline, including: the terminal performs image segmentation on the three-dimensional brain image to obtain the lesion area; based on the lesion area and the three-dimensional brain midline, determines the target area where the lesion is located, and the target area is the first half of the brain area or the second half of the brain area; the terminal removes the first contour surface in the target area from the lesion contour surface of the lesion area to obtain the remaining second contour surface; then determines two intersection lines of the lesion contour surface and the three-dimensional brain midline, removes the first brain midline between the two intersection lines in the three-dimensional brain midline, and obtains the remaining second brain midline; the terminal fits the second brain midline with the second contour surface to obtain the corrected three-dimensional brain midline.

[0280] In the embodiment of the present application, considering that the lesion will compress the brain midline if it has not broken into the ventricle, resulting in a severe deviation of the user's brain midline in actual situations, and the above-mentioned process of obtaining the three-dimensional brain midline through the brain segmentation model does not take into account the impact of the lesion on the brain midline, therefore, after obtaining the three-dimensional brain midline, the three-dimensional brain midline is corrected in combination with the brain region state, so that the final three-dimensional brain midline obtained can be more consistent with the user's actual situation, that is, the accuracy of the obtained three-dimensional brain midline is guaranteed. In addition, when correcting the three-dimensional brain midline, considering that the lesion usually compresses the three-dimensional brain midline toward the other hemisphere of the brain region opposite to the lesion, and the shape of the three-dimensional brain midline deviation is consistent with the shape of the lesion region contour, therefore, combining the partial lesion contour surface in the other hemisphere of the brain region and the partial three-dimensional brain midline at both ends of the two intersection lines of the lesion contour surface and the three-dimensional brain midline, refitting the three-dimensional brain midline, can ensure that the deviation shape of the three-dimensional brain midline is consistent with the user's actual brain midline shape, thus ensuring the accuracy of the final obtained brain midline.

[0281] Figure 11 A schematic diagram of a scan layer in a 3D brain image. Figure 11 Line 1101 and line 1102 can be seen on this scan layer. Line 1101 is the projection of the corrected 3D brain centerline on this scan layer, while line 1102 is the projection of the 3D brain centerline on this scan layer before correction. The brighter area on the second side of line 1101 in this scan layer is the lesion area. Figure 12 A schematic diagram of a scan layer in a 3D brain image. Figure 12 Line 1201 and line 1202 can be seen on this scan layer. Line 1201 is the projection of the 3D brain centerline on this scan layer before correction, and line 1202 is the projection of the 3D brain centerline on this scan layer after correction. The area with higher brightness on the first side of line 1202 in this scan layer is the lesion area.

[0282] In a possible implementation, the terminal performs image segmentation on the three-dimensional brain image to obtain the lesion area, including: the terminal calls a lesion segmentation model to perform image segmentation on the three-dimensional brain image to obtain the lesion area.

[0283] In one possible implementation, the terminal calls the lesion segmentation model, performs image segmentation on the three-dimensional brain image, and obtains the lesion area, including: the terminal calls the lesion segmentation model, performs image segmentation on the three-dimensional brain image, and obtains a lesion probability map. The lesion probability map is used to represent the probability that each point in the three-dimensional brain image belongs to the lesion area. Then, the terminal determines the points in the lesion probability map whose corresponding probabilities are greater than the second probability threshold as points in the lesion area, and determines the set of points in the lesion area as the lesion area. The second threshold probability is set to any value as needed, and the embodiments of the present application do not impose any restrictions on this.

[0284] In a possible implementation, the lesion segmentation model includes an encoding sub-model and a decoding sub-model. The encoding sub-model includes k encoding networks, and the decoding sub-model includes k decoding networks. Correspondingly, the terminal calls the lesion segmentation model to perform image segmentation on the three-dimensional brain image to obtain a lesion probability map, including: the terminal inputs the three-dimensional brain image into the first encoding network, extracts features of the three-dimensional brain image based on the first encoding network, and outputs the first encoding feature map; the terminal extracts features from the encoding feature map output by the previous network based on the i-th encoding network, outputs the i-th encoding feature map, until the k-th encoding network outputs the k-th encoding feature map, and inputs the k-th encoding feature map into the first decoding network; the terminal reconstructs the k-th encoding feature map based on the first decoding network to obtain the first decoding feature map; the terminal reconstructs the feature map after fusion of the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map based on the i-th decoding network, outputs the i-th decoding feature map, until the k-th decoding network outputs the k-th decoding feature map, and determines the k-th decoding feature map as the lesion probability map. Wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

[0285] In a possible implementation, the encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks. Accordingly, the terminal extracts features from the encoding feature map output by the previous network based on the i-th encoding network and outputs the i-th encoding feature map, including: the terminal downsamples the encoding feature map output by the i-1-th encoding network based on the i-1-th downsampling network, outputs the i-1-th intermediate encoding feature map, inputs the i-1-th intermediate encoding feature map to the i-th encoding network, extracts features from the i-1-th intermediate encoding feature map based on the i-th encoding network, and outputs the i-th encoding feature map; the terminal extracts features from the decoding feature map output by the previous network based on the i-th decoding network. The terminal upsamples the decoding feature map output by the i-1th decoding network based on the i-1th upsampling network, outputs the i-1th intermediate decoding feature map, inputs the i-1th intermediate decoding feature map to the i-th decoding network, and reconstructs the feature map after fusing the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map based on the i-th decoding network, and outputs the i-th decoding feature map.

[0286] It should be noted that the terminal calls the lesion segmentation model to perform image segmentation on the three-dimensional brain image to obtain the lesion probability map in the same way as the terminal calls the brain segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain the first half brain probability map and the second half brain probability map, which will not be repeated here.

[0287] Figure 14 Schematic diagram of the process of identifying the 3D brain midline. Figure 14 , obtain the original brain image, which is a three-dimensional image, and perform rigid registration on the original brain image to obtain a registered three-dimensional brain image. The three-dimensional brain image is input into the segmentation model and the classification model respectively, and the lesion segmentation map and the brain midline segmentation map are obtained through the segmentation model, wherein the lesion segmentation map has segmented the lesion area, and the midline segmentation image has segmented the brain midline. The classification model is used to determine the corresponding state of the brain area. When the lesion has broken into the ventricle, the brain midline obtained by the segmentation model is directly used as the final three-dimensional brain midline. When the lesion has not broken into the ventricle, the three-dimensional brain midline is corrected based on the lesion area in the lesion segmentation map to obtain the final brain midline.

[0288] Optionally, after obtaining the corrected three-dimensional brain midline, a standard three-dimensional brain midline of a relatively normal brain is obtained. Then, based on the corrected three-dimensional brain midline and the standard three-dimensional brain midline, the brain midline offset of the corrected three-dimensional brain midline can be determined, thereby determining the state of the user's brain based on the brain midline offset, and then measuring the user's brain disease.

[0289] One point that needs to be explained is that since the position and offset of the brain midline are often closely related to anatomical structures such as other soft tissues in the brain, these soft tissues (ventricles, hematomas, tumors, etc.) are often likely to have unclear anatomical structures due to low contrast. Therefore, it is difficult to fully grasp the shape and deformation of other anatomical structures in the brain by simply identifying the brain midline from a two-dimensional brain image, and thus it is impossible to accurately identify the brain midline. The embodiment of the present application utilizes a three-dimensional brain image to obtain complete three-dimensional structural information of various tissues in the brain area, so that the distribution of the three-dimensional brain midline can be judged as a whole, thereby obtaining an accurate three-dimensional brain midline. Moreover, compared with the brain midline identified from a two-dimensional brain image, the three-dimensional brain midline has a better visualization effect and can provide more information.

[0290] Another point that needs to be explained is that the above steps 506-507 are optional and are not performed in other embodiments.

[0291] In an embodiment of the present application, considering that the sample three-dimensional brain image used in training the brain segmentation model is bilaterally symmetrical, the template brain image is used to perform rigid registration on the original brain image so that the acquired three-dimensional brain image is bilaterally symmetrical. When the three-dimensional brain image is subsequently segmented by the brain segmentation model, the segmentation difficulty of the brain segmentation model can be reduced, thereby ensuring the accuracy of image segmentation performed by the brain segmentation model.

[0292] In an embodiment of the present application, by adjusting the window width and window level of the CT image, the clarity of the CT image can be adjusted, thereby reducing the difficulty of the brain segmentation model in performing image segmentation and ensuring the accuracy of the brain segmentation model in performing image segmentation.

[0293] In an embodiment of the present application, considering that the interval between any two scanning layers in the sample three-dimensional brain image used in training the brain segmentation model is a reference interval, the original brain image is sampled so that the interval between any two scanning layers in the obtained three-dimensional brain image is also a reference interval. When the three-dimensional brain image is subsequently segmented by the brain segmentation model, the segmentation difficulty of the brain segmentation model can be reduced, thereby ensuring the accuracy of image segmentation performed by the brain segmentation model.

[0294] In an embodiment of the present application, considering that the size of the sample three-dimensional brain image used in training the brain segmentation model is a reference size, the original brain image is sampled so that the size of the obtained three-dimensional brain image is also a reference size. When the three-dimensional brain image is subsequently segmented by the brain segmentation model, the segmentation difficulty of the brain segmentation model can be reduced, thereby ensuring the accuracy of image segmentation performed by the brain segmentation model.

[0295] In an embodiment of the present application, when performing image segmentation using a brain segmentation model to obtain a probability map of the first and second hemispheres, downsampling is performed by the downsampling network in the encoding sub-model to obtain low-resolution intermediate encoding features, thereby providing an identification basis for the first and second hemispheres. Upsampling is performed by the upsampling network in the decoding sub-model to obtain high-resolution intermediate decoding features, thereby providing a positioning basis for the precise segmentation of the first and second hemispheres. This ensures the accuracy of the probability map of the first and second hemispheres determined by the brain segmentation model. In addition, when performing image reconstruction, the decoding network following the first decoding network fills the underlying information in the decoding feature map by fusing the encoding feature map output by the encoding network with the decoding feature map output by the decoding network, thereby improving the segmentation accuracy of the brain segmentation model.

[0296] In an embodiment of the present application, a three-dimensional brain image is segmented using a brain segmentation model to obtain a first-half brain probability map and a second-half brain probability map, and then the first-half brain region and the second-half brain region are determined by threshold segmentation. The determination of the first-half brain region and the second-half brain region is efficient and accurate.

[0297] In the embodiment of the present application, considering that the lesion will compress the brain midline if it has not broken into the ventricle, resulting in a severe deviation of the user's brain midline in actual situations, and the above-mentioned process of obtaining the three-dimensional brain midline through the brain segmentation model does not take into account the impact of the lesion on the brain midline, therefore, after obtaining the three-dimensional brain midline, the three-dimensional brain midline is corrected in combination with the brain region state, so that the final three-dimensional brain midline obtained can be more consistent with the user's actual situation, that is, the accuracy of the obtained three-dimensional brain midline is guaranteed. In addition, when correcting the three-dimensional brain midline, considering that the lesion usually compresses the three-dimensional brain midline toward the other hemisphere of the brain region opposite to the lesion, and the shape of the three-dimensional brain midline deviation is consistent with the shape of the lesion region contour, therefore, combining the partial lesion contour surface in the other hemisphere of the brain region and the partial three-dimensional brain midline at both ends of the two intersection lines of the lesion contour surface and the three-dimensional brain midline, refitting the three-dimensional brain midline, can ensure that the deviation shape of the three-dimensional brain midline is consistent with the user's actual brain midline shape, thus ensuring the accuracy of the final obtained brain midline.

[0298] Figure 15 This is a flow chart of a brain midline recognition method provided in an embodiment of the present application. This embodiment describes the training process of the lesion segmentation model. Figure 15 , the embodiment includes:

[0299] 1501. The terminal obtains third sample data, where the third sample data includes a sample three-dimensional brain image and a second sample segmented image, where the second sample segmented image is marked with a lesion area in the sample three-dimensional brain image.

[0300] 1502. The terminal calls the lesion segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain a lesion probability map.

[0301] The lesion probability map is used to represent the probability that each point in the sample three-dimensional brain image belongs to the lesion area.

[0302] In a possible implementation, the lesion segmentation model includes an encoding sub-model and a decoding sub-model. The encoding sub-model includes k encoding networks, and the decoding sub-model includes k decoding networks. Correspondingly, the terminal calls the lesion segmentation model, performs image segmentation on the sample three-dimensional brain image, and obtains a lesion probability map, including: the terminal inputs the sample three-dimensional brain image into the first encoding network, performs feature extraction on the sample three-dimensional brain image based on the first encoding network, and outputs the first encoding feature map; the terminal extracts features from the encoding feature map output by the previous network based on the i-th encoding network, outputs the i-th encoding feature map, until the k-th encoding network outputs the k-th encoding feature map, and inputs the k-th encoding feature map into the first decoding network; the terminal reconstructs the k-th encoding feature map based on the first decoding network to obtain the first decoding feature map; the terminal reconstructs the feature map after fusion of the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map based on the i-th decoding network, outputs the i-th decoding feature map, until the k-th decoding network outputs the k-th decoding feature map, and determines the k-th decoding feature map as the lesion probability map. Wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

[0303] In a possible implementation, the encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks. Accordingly, the terminal extracts features from the encoding feature map output by the previous network based on the i-th encoding network and outputs the i-th encoding feature map, including: the terminal downsamples the encoding feature map output by the i-1-th encoding network based on the i-1-th downsampling network, outputs the i-1-th intermediate encoding feature map, inputs the i-1-th intermediate encoding feature map to the i-th encoding network, extracts features from the i-1-th intermediate encoding feature map based on the i-th encoding network, and outputs the i-th encoding feature map; the terminal extracts features from the decoding feature map output by the previous network based on the i-th decoding network. The terminal upsamples the decoding feature map output by the i-1th decoding network based on the i-1th upsampling network, outputs the i-1th intermediate decoding feature map, inputs the i-1th intermediate decoding feature map to the i-th decoding network, and reconstructs the feature map after fusing the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map based on the i-th decoding network, and outputs the i-th decoding feature map.

[0304] 1503. The terminal trains a lesion segmentation model based on the lesion probability map and the second sample segmentation image.

[0305] In one possible implementation, a lesion segmentation model is trained based on the lesion probability map and the second sample segmentation image, including: the terminal determines at least one point belonging to the lesion area based on the second sample segmentation image; determines the probability that at least one point belonging to the lesion area belongs to the lesion area based on the lesion probability map; determines the loss value of the lesion segmentation model based on the probability that at least one point belonging to the lesion area belongs to the lesion area; and trains the lesion segmentation model based on the loss value.

[0306] It should be noted that the implementation of steps 1501-1503 is similar to that of steps 301-303, and will not be repeated here.

[0307] In an embodiment of the present application, when performing image segmentation through the lesion segmentation model to obtain a lesion probability map, downsampling is performed through the downsampling network in the encoding sub-model to obtain low-resolution intermediate encoding features, thereby providing a basis for lesion identification, and upsampling is performed through the upsampling network in the decoding sub-model to obtain high-resolution intermediate decoding features, thereby providing a positioning basis for accurate segmentation of the lesion, thereby ensuring the accuracy of the lesion probability map determined by the brain segmentation model. In addition, when the decoding network after the first decoding network performs image reconstruction, the decoding feature map output by the encoding network is fused with the decoding feature map output by the decoding network to fill in the underlying information in the decoding feature map, thereby improving the segmentation accuracy of the brain segmentation model.

[0308] It should be noted that the above method is only described as an example in which the lesion segmentation model and the brain segmentation model are different segmentation models. In other embodiments, the lesion segmentation model and the brain segmentation model are the same segmentation model, and the segmentation model not only has the function of segmenting the first half brain region and the second half brain region, but also has the function of segmenting the lesion region. Optionally, when the lesion segmentation model and the brain segmentation model are the same segmentation model, the segmentation model is trained by the fourth sample data, wherein the fourth sample data includes a sample three-dimensional brain image and a third sample segmentation image, and the third sample segmentation image is marked with the first half brain region, the second half brain region and the lesion region in the sample three-dimensional brain image. After the segmentation model is trained by the fourth sample data, the segmentation model can be called to perform image segmentation on any three-dimensional brain image to obtain the first half brain region, the second half brain region and the lesion region. It should be noted that the implementation method of training the segmentation model by the fourth sample data is similar to the implementation method of training the brain segmentation model by the first sample data. The method of obtaining the first half of the brain region, the second half of the brain region and the lesion region in any three-dimensional brain image through this segmentation model is the same as the method of obtaining the first half of the brain region and the second half of the brain region in any three-dimensional brain image through the above-mentioned brain segmentation model, and will not be repeated here.

[0309] In the embodiment of the present application, since segmenting the first hemisphere, the second hemisphere, or the lesion region in a three-dimensional brain image region is a related image segmentation task, the segmentation model is trained using a multi-task learning machine learning method, enabling the segmentation model to achieve these three functions, thereby improving the model's training efficiency. Multi-task learning is a machine learning method that learns multiple related subtasks within the same model.

[0310] It should be noted that, optionally, the brain segmentation model and lesion segmentation model in this application adopt 3DUnet (a neural network model), FCN (Fully Convolutional Networks) or other neural networks, and this application does not impose any restrictions on this. Optionally, when performing image segmentation, a non-learning algorithm is used for image segmentation, and a non-learning algorithm refers to an unsupervised machine learning algorithm.

[0311] All of the above optional technical solutions can be combined in any way to form optional embodiments of the present application, and will not be described in detail here.

[0312] Figure 16 This is a block diagram of a brain midline recognition device provided by an embodiment of the present application. Figure 16 , the device comprises:

[0313] An image acquisition module 161 is used to acquire a three-dimensional brain image, where the three-dimensional brain image includes a brain region;

[0314] A first recognition module 162 is configured to recognize a contour surface of a brain region from a three-dimensional brain image;

[0315] An image segmentation module 163 is used to segment the three-dimensional brain image to obtain a first hemisphere region and a second hemisphere region;

[0316] A second recognition module 164 is configured to recognize a first hemisphere contour surface of the first hemisphere region and a second hemisphere contour surface of the second hemisphere region from the three-dimensional brain image;

[0317] The brain midline determination module 165 is configured to determine the three-dimensional brain midline of the brain region by taking the difference between the union of the first half brain contour surface and the second half brain contour surface and the contour surface of the brain region.

[0318] In a possible implementation, the image segmentation module 163 is configured to call a brain segmentation model to perform image segmentation on a three-dimensional brain image to obtain a first hemisphere region and a second hemisphere region.

[0319] In one possible implementation, see Figure 17 , the image segmentation module 163 includes:

[0320] a first segmentation unit 1631 configured to call a brain segmentation model to perform image segmentation on the three-dimensional brain image to obtain a first hemisphere probability map and a second hemisphere probability map, wherein the first hemisphere probability map indicates the probability that each point in the three-dimensional brain image belongs to the first hemisphere region, and the second hemisphere probability map indicates the probability that each point in the three-dimensional brain image belongs to the second hemisphere region;

[0321] Region determination unit 1632 is configured to determine, in the first hemisphere probability map, points whose corresponding probabilities are greater than a first probability threshold as points in the first hemisphere region, and to determine the set of points in the first hemisphere region as the first hemisphere region; and to determine, in the second hemisphere probability map, points whose corresponding probabilities are greater than the first probability threshold as points in the second hemisphere region, and to determine the set of points in the second hemisphere region as the second hemisphere region.

[0322] In one possible implementation, the brain segmentation model includes an encoding sub-model and a decoding sub-model, the encoding sub-model includes k encoding networks, and the decoding sub-model includes k decoding networks;

[0323] The first segmentation unit 1631 is used to input the three-dimensional brain image into the first encoding network, perform feature extraction on the three-dimensional brain image based on the first encoding network, and output the first encoding feature map; based on the i-th encoding network, perform feature extraction on the encoding feature map output by the previous network, output the i-th encoding feature map, until the k-th encoding network outputs the k-th encoding feature map, and input the k-th encoding feature map into the first decoding network; based on the first decoding network, perform image reconstruction on the k-th encoding feature map to obtain the first decoding feature map; based on the i-th decoding network, perform image reconstruction on the feature map after fusion of the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map, output the i-th decoding feature map, until the k-th decoding network outputs the k-th decoding feature map, and the k-th decoding feature map includes the first half-brain probability map and the second half-brain probability map; wherein k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

[0324] In a possible implementation, the encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks;

[0325] The first segmentation unit 1631 is used to downsample the encoding feature map output by the i-1th encoding network based on the i-1th downsampling network, output the i-1th intermediate encoding feature map, input the i-1th intermediate encoding feature map to the i-th encoding network, perform feature extraction on the i-1th intermediate encoding feature map based on the i-th encoding network, and output the i-th encoding feature map; upsample the decoding feature map output by the i-1th decoding network based on the i-1th upsampling network, output the i-1th intermediate decoding feature map, input the i-1th intermediate decoding feature map to the i-th decoding network, reconstruct the feature map after the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map are fused based on the i-1th decoding network, and output the i-th decoding feature map.

[0326] In one possible implementation, the training process of the brain segmentation model includes:

[0327] Acquire first sample data, the first sample data including a sample three-dimensional brain image and a first sample segmented image, wherein the first sample segmented image is marked with a first hemisphere region and a second hemisphere region in the sample three-dimensional brain image;

[0328] Calling the brain segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain a first-half brain probability map and a second-half brain probability map. The first-half brain probability map is used to represent the probability that each point in the sample three-dimensional brain image belongs to the first-half brain region, and the second-half brain probability map is used to represent the probability that each point in the sample three-dimensional brain image belongs to the second-half brain region.

[0329] A brain segmentation model is trained based on the first hemisphere probability map, the second hemisphere probability map, and the first sample segmentation image.

[0330] In one possible implementation, training a brain segmentation model based on the first brain hemisphere probability map, the second brain hemisphere probability map, and the first sample segmentation image includes:

[0331] Determining, based on the first sample segmented image, at least one point belonging to the first hemisphere region and at least one point belonging to the second hemisphere region;

[0332] Determining, based on the first hemisphere probability map, a probability that at least one point belonging to the first hemisphere region belongs to the first hemisphere region, and determining, based on the second hemisphere probability map, a probability that at least one point belonging to the second hemisphere region belongs to the second hemisphere region;

[0333] determining a loss value of a brain segmentation model based on a probability that at least one point belonging to the first hemisphere region belongs to the first hemisphere region and a probability that at least one point belonging to the second hemisphere region belongs to the second hemisphere region;

[0334] Based on the loss value, train the brain segmentation model.

[0335] In one possible implementation, the brain segmentation model includes an encoding sub-model and a decoding sub-model, the encoding sub-model includes k encoding networks, and the decoding sub-model includes k decoding networks;

[0336] Call the brain segmentation model to perform image segmentation on the sample 3D brain image to obtain the first half brain probability map and the second half brain probability map, including:

[0337] Inputting the sample three-dimensional brain image into the first encoding network, performing feature extraction on the sample three-dimensional brain image based on the first encoding network, and outputting the first encoding feature map;

[0338] Based on the i-th encoding network, feature extraction is performed on the encoding feature map output by the previous network, and the i-th encoding feature map is output, until the k-th encoding network outputs the k-th encoding feature map, and the k-th encoding feature map is input into the first decoding network;

[0339] Reconstruct the kth encoding feature map based on the first decoding network to obtain the first decoding feature map;

[0340] Based on the i-th decoding network, the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map are fused to perform image reconstruction, and the i-th decoding feature map is output until the k-th decoding network outputs the k-th decoding feature map. The k-th decoding feature map includes the first hemisphere probability map and the second hemisphere probability map;

[0341] Wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

[0342] In one possible implementation, the encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks; based on the i-th encoding network, feature extraction is performed on the encoding feature map output by the previous network, and the i-th encoding feature map is output, including:

[0343] Based on the i-1th downsampling network, downsample the encoding feature map output by the i-1th encoding network, output the i-1th intermediate encoding feature map, input the i-1th intermediate encoding feature map to the i-th encoding network, and based on the i-th encoding network, perform feature extraction on the i-1th intermediate encoding feature map and output the i-th encoding feature map;

[0344] Based on the i-th decoding network, the feature map after the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map is fused to perform image reconstruction, and the i-th decoding feature map is output, including:

[0345] Based on the i-1th upsampling network, the decoding feature map output by the i-1th decoding network is upsampled, and the i-1th intermediate decoding feature map is output. The i-1th intermediate decoding feature map is input to the i-th decoding network. Based on the i-th decoding network, the feature map after the fusion of the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map is reconstructed, and the i-th decoding feature map is output.

[0346] In a possible implementation, the first recognition module 162 is configured to recognize contour points of a brain region from a three-dimensional brain image; and determine a set of contour points of the brain region as a contour surface of the brain region;

[0347] The second recognition module 164 is used to identify the contour points of the first hemisphere region and the contour points of the second hemisphere region from the three-dimensional brain image; determine the set of contour points of the first hemisphere region as the first hemisphere contour surface, and determine the set of contour points of the second hemisphere region as the second hemisphere contour surface.

[0348] In one possible implementation, see Figure 17 , the device further comprises:

[0349] a state determination module 166 for determining a state corresponding to a brain region based on the three-dimensional brain image, the state being whether the lesion has broken into the ventricle or the lesion has not broken into the ventricle;

[0350] The brain centerline correction module 167 is configured to correct the three-dimensional brain centerline in response to the status that the lesion has not broken into the ventricle.

[0351] In one possible implementation, see Figure 17 , the brain midline correction module 167, including:

[0352] The second segmentation unit 1671 is used to segment the three-dimensional brain image to obtain the lesion area;

[0353] The brain midline correction unit 1672 is used to determine the target area where the lesion is located based on the lesion area and the three-dimensional brain midline, where the target area is the first hemisphere area or the second hemisphere area; remove the first contour surface in the target area from the lesion contour surface of the lesion area to obtain the remaining second contour surface; determine the two intersection lines of the lesion contour surface and the three-dimensional brain midline, remove the first brain midline between the two intersection lines in the three-dimensional brain midline to obtain the remaining second brain midline; fit the second brain midline with the second contour surface to obtain the corrected three-dimensional brain midline.

[0354] In one possible implementation, the state determination module 166 is used to call a classification model to classify the three-dimensional brain image to obtain a classification label. The classification label is used to describe the state corresponding to the brain region, where the state is whether the lesion has broken into the ventricle or the lesion has not broken into the ventricle.

[0355] In one possible implementation, the classification model training process includes:

[0356] Acquire second sample data, the second sample data including a sample three-dimensional brain image and a sample label, the sample three-dimensional brain image including a brain region, and the sample label describing a state corresponding to the brain region, where the state is either the lesion has ruptured into the ventricle or the lesion has not ruptured into the ventricle;

[0357] Call the classification model to classify the sample 3D brain image and obtain the predicted label;

[0358] Based on the sample labels and predicted labels, a classification model is trained.

[0359] In a possible implementation, the second segmentation unit 1671 is configured to call a lesion segmentation model to perform image segmentation on a three-dimensional brain image to obtain a lesion area.

[0360] In one possible implementation, see Figure 17 The second segmentation unit 1671 includes:

[0361] The segmentation subunit 16711 is used to call the lesion segmentation model to perform image segmentation on the three-dimensional brain image to obtain a lesion probability map. The lesion probability map is used to represent the probability that each point in the three-dimensional brain image belongs to the lesion area;

[0362] The determination subunit 16712 is configured to determine points in the lesion probability map whose corresponding probabilities are greater than a second probability threshold as points in the lesion area, and determine a set of points in the lesion area as the lesion area.

[0363] In one possible implementation, the lesion segmentation model includes an encoding sub-model and a decoding sub-model, the encoding sub-model includes k encoding networks, and the decoding sub-model includes k decoding networks;

[0364] The segmentation subunit 16711 is used to input the three-dimensional brain image into the first encoding network, perform feature extraction on the three-dimensional brain image based on the first encoding network, and output the first encoding feature map; based on the i-th encoding network, perform feature extraction on the encoding feature map output by the previous network, output the i-th encoding feature map, until the k-th encoding network outputs the k-th encoding feature map, and input the k-th encoding feature map into the first decoding network; based on the first decoding network, perform image reconstruction on the k-th encoding feature map to obtain the first decoding feature map; based on the i-th decoding network, perform image reconstruction on the feature map after fusion of the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map, output the i-th decoding feature map, until the k-th decoding network outputs the k-th decoding feature map, and determine the k-th decoding feature map as the lesion probability map; wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

[0365] In a possible implementation, the encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks;

[0366] The segmentation subunit 16711 is used to downsample the encoding feature map output by the i-1th encoding network based on the i-1th downsampling network, output the i-1th intermediate encoding feature map, input the i-1th intermediate encoding feature map to the i-th encoding network, perform feature extraction on the i-1th intermediate encoding feature map based on the i-th encoding network, and output the i-th encoding feature map; upsample the decoding feature map output by the i-1th decoding network based on the i-1th upsampling network, output the i-1th intermediate decoding feature map, input the i-1th intermediate decoding feature map to the i-th decoding network, perform image reconstruction on the feature map after fusing the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map based on the i-th decoding network, and output the i-th decoding feature map.

[0367] In one possible implementation, the training process of the lesion segmentation model includes:

[0368] Acquire third sample data, the third sample data including the sample three-dimensional brain image and the second sample segmented image, wherein the second sample segmented image is marked with a lesion area in the sample three-dimensional brain image;

[0369] Calling the lesion segmentation model to perform image segmentation on the sample 3D brain image to obtain a lesion probability map. The lesion probability map is used to represent the probability that each point in the sample 3D brain image belongs to the lesion area;

[0370] Based on the lesion probability map and the second sample segmentation image, a lesion segmentation model is trained.

[0371] In one possible implementation, training a lesion segmentation model based on the lesion probability map and the second sample segmentation image includes:

[0372] determining at least one point belonging to the lesion area based on the second sample segmented image;

[0373] determining, based on the lesion probability map, a probability that at least one point belonging to the lesion region belongs to the lesion region;

[0374] Determining a loss value of a lesion segmentation model based on a probability that at least one point belonging to the lesion region belongs to the lesion region;

[0375] Based on the loss value, the lesion segmentation model is trained.

[0376] In one possible implementation, the lesion segmentation model includes an encoding sub-model and a decoding sub-model, the encoding sub-model includes k encoding networks, and the decoding sub-model includes k decoding networks;

[0377] Call the lesion segmentation model to perform image segmentation on the sample 3D brain image to obtain the lesion probability map, including:

[0378] Inputting the sample three-dimensional brain image into the first encoding network, performing feature extraction on the sample three-dimensional brain image based on the first encoding network, and outputting the first encoding feature map;

[0379] Based on the i-th encoding network, feature extraction is performed on the encoding feature map output by the previous network, and the i-th encoding feature map is output, until the k-th encoding network outputs the k-th encoding feature map, and the k-th encoding feature map is input into the first decoding network;

[0380] Reconstruct the kth encoding feature map based on the first decoding network to obtain the first decoding feature map;

[0381] Based on the i-th decoding network, the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map are fused to perform image reconstruction, and the i-th decoding feature map is output, until the k-th decoding network outputs the k-th decoding feature map, and the k-th decoding feature map is determined as the lesion probability map;

[0382] Wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

[0383] In one possible implementation, the encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks; based on the i-th encoding network, feature extraction is performed on the encoding feature map output by the previous network, and the i-th encoding feature map is output, including:

[0384] Based on the i-1th downsampling network, downsample the encoding feature map output by the i-1th encoding network, output the i-1th intermediate encoding feature map, input the i-1th intermediate encoding feature map to the i-th encoding network, and based on the i-th encoding network, perform feature extraction on the i-1th intermediate encoding feature map and output the i-th encoding feature map;

[0385] Based on the i-th decoding network, the feature map after the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map is fused to perform image reconstruction, and the i-th decoding feature map is output, including:

[0386] Based on the i-1th upsampling network, the decoding feature map output by the i-1th decoding network is upsampled, and the i-1th intermediate decoding feature map is output. The i-1th intermediate decoding feature map is input to the i-th decoding network. Based on the i-th decoding network, the feature map after the fusion of the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map is reconstructed, and the i-th decoding feature map is output.

[0387] In one possible implementation, the image acquisition module 161 is used to acquire an original brain image, which is a three-dimensional image; acquire a template brain image, which is bilaterally symmetrical; and perform rigid registration on the original brain image according to the template brain image to obtain a three-dimensional brain image.

[0388] In one possible implementation, the image acquisition module 161 is used to acquire a computed tomography (CT) image of the brain, and determine the window width and window position of the CT image as a reference window width and a reference window position, respectively, to obtain a three-dimensional brain image; wherein the window width represents the range of CT values ​​displayed in the CT image, and the window position represents the middle value of the range of CT values.

[0389] In one possible implementation, the image acquisition module 161 is configured to acquire an original brain image, where the original brain image includes multiple scan layers; sample the multiple scan layers so that the interval between any two scan layers reaches a reference interval; and use the sampled original brain image as a three-dimensional brain image.

[0390] In a possible implementation, the image acquisition module 161 is configured to acquire an original brain image, which is a three-dimensional image; and adjust the size of the original brain image to a reference size to obtain a three-dimensional brain image.

[0391] When identifying the brain midline, the embodiment of the present application converts the problem of identifying the brain midline into the problem of dividing the left and right brain halves, and uses a three-dimensional brain image to divide the left and right brains, thereby obtaining the first half brain contour surface and the second half brain contour surface. The difference between the union of the first half brain contour surface and the second half brain contour surface and the whole brain contour surface is the three-dimensional brain midline. Compared with the brain midline identified by a two-dimensional brain image, this three-dimensional brain midline can provide more information and is more conducive to determining the user's brain state.

[0392] It should be noted that the brain midline recognition device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate the recognition of the three-dimensional brain midline. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the brain midline recognition device provided in the above embodiment and the brain midline recognition method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0393] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed in the brain midline recognition method of the above embodiment.

[0394] Optionally, the computer device is provided as a terminal. Figure 18 The following is a block diagram of a terminal 1800 according to an exemplary embodiment of the present application. Terminal 1800 may be a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. Terminal 1800 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other similar names.

[0395] The terminal 1800 includes a processor 1801 and a memory 1802 .

[0396] The processor 1801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 1801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0397] Memory 1802 may include one or more computer-readable storage media, which may be non-transitory. Memory 1802 may also include high-speed random access memory and non-volatile memory, such as one or more magnetic disk storage devices or flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in memory 1802 is used to store at least one computer program, which is executed by processor 1801 to implement the brain midline identification method provided in the method embodiments of this application.

[0398] In some embodiments, terminal 1800 may optionally include a peripheral device interface 1803 and at least one peripheral device. The processor 1801, memory 1802, and peripheral device interface 1803 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 1803 via a bus, signal lines, or circuit boards. Specifically, the peripheral device may include at least one of a radio frequency circuit 1804, a display screen 1805, a camera assembly 1806, an audio circuit 1807, and a power supply 1809.

[0399] The peripheral device interface 1803 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 1801 and the memory 1802. In some embodiments, the processor 1801, the memory 1802, and the peripheral device interface 1803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1801, the memory 1802, and the peripheral device interface 1803 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0400] The RF circuit 1804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1804 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the RF circuit 1804 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The RF circuit 1804 can communicate with other terminals via at least one wireless communication protocol. Such wireless communication protocols include, but are not limited to, metropolitan area networks, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1804 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0401] The display screen 1805 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1805 is a touch screen display, the display screen 1805 also has the ability to collect touch signals on the surface or above the surface of the display screen 1805. The touch signal can be input as a control signal to the processor 1801 for processing. At this time, the display screen 1805 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, there can be one display screen 1805, which is set on the front panel of the terminal 1800; in other embodiments, there can be at least two display screens 1805, which are respectively set on different surfaces of the terminal 1800 or in a folding design; in other embodiments, the display screen 1805 can be a flexible display screen, which is set on the curved surface or folding surface of the terminal 1800. Even more, the display screen 1805 can be set as a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1805 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0402] The camera assembly 1806 is used to capture images or videos. Optionally, the camera assembly 1806 includes a front camera and a rear camera. The front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1806 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. The dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0403] The audio circuit 1807 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input into the processor 1801 for processing, or input into the radio frequency circuit 1804 to achieve voice communication. For the purpose of stereo sound collection or noise reduction, there may be multiple microphones, each located in different parts of the terminal 1800. The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert electrical signals from the processor 1801 or the radio frequency circuit 1804 into sound waves. The speaker may be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for purposes such as distance measurement. In some embodiments, the audio circuit 1807 may also include a headphone jack.

[0404] Power supply 1809 is used to power various components in terminal 1800. Power supply 1809 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1809 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0405] In some embodiments, the terminal 1800 further includes one or more sensors 1810 , including but not limited to: an acceleration sensor 1811 , a gyroscope sensor 1812 , a pressure sensor 1813 , an optical sensor 1815 , and a proximity sensor 1816 .

[0406] The accelerometer 1811 can detect the magnitude of acceleration along the three coordinate axes of the coordinate system established by the terminal 1800. For example, the accelerometer 1811 can be used to detect the components of gravity acceleration along the three coordinate axes. The processor 1801 can control the display screen 1805 to display the user interface in either a landscape or portrait view based on the gravity acceleration signal collected by the accelerometer 1811. The accelerometer 1811 can also be used to collect game or user motion data.

[0407] The gyroscope sensor 1812 can detect the orientation and rotation angle of the terminal 1800. It can work in conjunction with the accelerometer 1811 to collect the user's 3D movements on the terminal 1800. Based on the data collected by the gyroscope sensor 1812, the processor 1801 can implement the following functions: motion sensing (for example, changing the UI based on the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0408] The pressure sensor 1813 can be located on the side frame of the terminal 1800 and / or below the display screen 1805. When the pressure sensor 1813 is located on the side frame of the terminal 1800, it can detect the user's gripping signal of the terminal 1800. The processor 1801 then performs left-hand or right-hand recognition or shortcut operations based on the gripping signal collected by the pressure sensor 1813. When the pressure sensor 1813 is located below the display screen 1805, the processor 1801 controls the operable controls on the UI interface based on the user's pressure operation on the display screen 1805. Operable controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0409] Optical sensor 1815 is used to detect ambient light intensity. In one embodiment, processor 1801 can control the display brightness of display screen 1805 based on the ambient light intensity detected by optical sensor 1815. Specifically, when the ambient light intensity is high, the display brightness of display screen 1805 is increased; when the ambient light intensity is low, the display brightness of display screen 1805 is decreased. In another embodiment, processor 1801 can also dynamically adjust the shooting parameters of camera assembly 1806 based on the ambient light intensity detected by optical sensor 1815.

[0410] Proximity sensor 1816, also known as a distance sensor, is provided on the front panel of terminal 1800. Proximity sensor 1816 is used to detect the distance between the user and the front of terminal 1800. In one embodiment, when proximity sensor 1816 detects that the distance between the user and the front of terminal 1800 is gradually decreasing, processor 1801 controls display screen 1805 to switch from the screen-on state to the screen-off state. When proximity sensor 1816 detects that the distance between the user and the front of terminal 1800 is gradually increasing, processor 1801 controls display screen 1805 to switch from the screen-off state to the screen-on state.

[0411] Those skilled in the art will understand that Figure 18 The structure shown in the figure does not constitute a limitation on the terminal 1800, and the terminal 1800 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0412] Optionally, the computer device is provided as a server. Figure 10 1 is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server 1000 may vary significantly due to different configurations or performance, and may include one or more processors (central processing units, CPUs) 1001 and one or more memories 1002, wherein the memories 1002 store at least one computer program, which is loaded and executed by the processor 1001 to implement the brain midline recognition methods provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and input and output interfaces for input and output. The server may also include other components for implementing device functions, which will not be described in detail here.

[0413] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one computer program. The at least one computer program is loaded and executed by a processor to implement the operations performed in the brain midline recognition method of the above embodiment.

[0414] The present application also provides a computer program product or computer program, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the operations performed in the brain midline identification method in the various optional implementations described above.

[0415] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0416] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A brain midline identification method, characterized in that: The method comprises: acquiring a three-dimensional brain image, wherein the three-dimensional brain image includes a brain region; identifying a contour surface of the brain region from the three-dimensional brain image; Performing image segmentation on the three-dimensional brain image to obtain a first hemisphere region and a second hemisphere region; identifying a first hemisphere outline surface of the first hemisphere region and a second hemisphere outline surface of the second hemisphere region from the three-dimensional brain image; Determining the difference between the union of the first half brain contour surface and the second half brain contour surface and the contour surface of the brain region as the three-dimensional brain midline of the brain region; determining, based on the three-dimensional brain image, a state corresponding to the brain region, the state being whether the lesion has ruptured into the ventricle or the lesion has not ruptured into the ventricle; In response to the status being that the lesion has not broken into the ventricle, performing image segmentation on the three-dimensional brain image to obtain a lesion region, and determining a target region where the lesion is located based on the lesion region and the three-dimensional brain midline, the target region being the first hemisphere region or the second hemisphere region; removing a first contour surface in the target area from the lesion contour surface of the lesion area to obtain a second contour surface, determining two intersection lines between the lesion contour surface and the three-dimensional brain midline, and removing the first brain midline between the two intersection lines from the three-dimensional brain midline to obtain a remaining second brain midline; The second brain midline is fitted with the second contour surface to obtain a corrected three-dimensional brain midline.

2. The method according to claim 1, characterized in that The step of performing image segmentation on the three-dimensional brain image to obtain a first hemisphere region and a second hemisphere region includes: A brain segmentation model is called to perform image segmentation on the three-dimensional brain image to obtain the first hemisphere region and the second hemisphere region.

3. The method according to claim 2, characterized in that The calling of the brain segmentation model to perform image segmentation on the three-dimensional brain image to obtain the first hemisphere region and the second hemisphere region includes: Invoking the brain segmentation model to perform image segmentation on the three-dimensional brain image to obtain a first-half brain probability map and a second-half brain probability map, wherein the first-half brain probability map is used to represent the probability that each point in the three-dimensional brain image belongs to the first-half brain region, and the second-half brain probability map is used to represent the probability that each point in the three-dimensional brain image belongs to the second-half brain region; Determine, in the first hemisphere probability map, points whose corresponding probabilities are greater than a first probability threshold as points in the first hemisphere region, and determine a set of points in the first hemisphere region as the first hemisphere region; In the second hemisphere probability map, points corresponding to probabilities greater than the first probability threshold are determined as points in the second hemisphere region, and a set of points in the second hemisphere region is determined as the second hemisphere region.

4. The method according to claim 3, characterized in that The brain segmentation model includes an encoding sub-model and a decoding sub-model, wherein the encoding sub-model includes k encoding networks and the decoding sub-model includes k decoding networks; The calling of the brain segmentation model to perform image segmentation on the three-dimensional brain image to obtain a first hemisphere probability map and a second hemisphere probability map includes: Inputting the three-dimensional brain image into a first encoding network, performing feature extraction on the three-dimensional brain image based on the first encoding network, and outputting a first encoding feature map; Based on the i-th encoding network, feature extraction is performed on the encoding feature map output by the previous network, and the i-th encoding feature map is output, until the k-th encoding network outputs the k-th encoding feature map, and the k-th encoding feature map is input into the first decoding network; Reconstructing the k-th encoding feature map based on the first decoding network to obtain a first decoding feature map; Based on the i-th decoding network, reconstruct the feature map after fusing the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map, and output the i-th decoding feature map until the k-th decoding network outputs the k-th decoding feature map, wherein the k-th decoding feature map includes the first hemi-brain probability map and the second hemi-brain probability map; Wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

5. The method according to claim 4, characterized in that The encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks; the step of extracting features from the encoding feature map output by the previous network based on the i-th encoding network and outputting the i-th encoding feature map comprises: Based on the i-1th downsampling network, downsampling the encoding feature map output by the i-1th encoding network, outputting the i-1th intermediate encoding feature map, inputting the i-1th intermediate encoding feature map into the i-th encoding network, performing feature extraction on the i-1th intermediate encoding feature map based on the i-th encoding network, and outputting the i-th encoding feature map; The method comprises: reconstructing an image of a feature map obtained by fusing a decoding feature map output by a previous network and a (k+1-i)th encoding feature map based on the i-th decoding network, and outputting the i-th decoding feature map. Based on the i-1th upsampling network, the decoding feature map output by the i-1th decoding network is upsampled, and the i-1th intermediate decoding feature map is output. The i-1th intermediate decoding feature map is input into the i-th decoding network. Based on the i-th decoding network, the feature map after the fusion of the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map is reconstructed, and the i-th decoding feature map is output.

6. The method according to claim 3, characterized in that The training process of the brain segmentation model includes: Acquire first sample data, the first sample data including a sample three-dimensional brain image and a first sample segmented image, wherein the first sample segmented image is annotated with a first hemisphere region and a second hemisphere region in the sample three-dimensional brain image; Invoking the brain segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain a first half-brain probability map and a second half-brain probability map, wherein the first half-brain probability map is used to represent the probability that each point in the sample three-dimensional brain image belongs to the first half-brain region, and the second half-brain probability map is used to represent the probability that each point in the sample three-dimensional brain image belongs to the second half-brain region; The brain segmentation model is trained based on the first brain hemisphere probability map, the second brain hemisphere probability map, and the first sample segmentation image.

7. The method according to claim 6, characterized in that The training of the brain segmentation model based on the first brain hemisphere probability map, the second brain hemisphere probability map, and the first sample segmentation image includes: determining, based on the first sample segmented image, at least one point belonging to the first hemisphere region and at least one point belonging to the second hemisphere region; Determining, based on the first hemisphere probability map, a probability that the at least one point belonging to the first hemisphere region belongs to the first hemisphere region, and determining, based on the second hemisphere probability map, a probability that the at least one point belonging to the second hemisphere region belongs to the second hemisphere region; determining a loss value of the brain segmentation model based on a probability that the at least one point belonging to the first hemisphere region belongs to the first hemisphere region and a probability that the at least one point belonging to the second hemisphere region belongs to the second hemisphere region; The brain segmentation model is trained based on the loss value.

8. The method according to claim 6, characterized in that The brain segmentation model includes an encoding sub-model and a decoding sub-model, wherein the encoding sub-model includes k encoding networks and the decoding sub-model includes k decoding networks; The calling of the brain segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain a first half brain probability map and a second half brain probability map includes: Inputting the sample three-dimensional brain image into a first encoding network, performing feature extraction on the sample three-dimensional brain image based on the first encoding network, and outputting a first encoding feature map; Based on the i-th encoding network, feature extraction is performed on the encoding feature map output by the previous network, and the i-th encoding feature map is output, until the k-th encoding network outputs the k-th encoding feature map, and the k-th encoding feature map is input into the first decoding network; Reconstructing the k-th encoding feature map based on the first decoding network to obtain a first decoding feature map; Based on the i-th decoding network, reconstruct the feature map after fusing the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map, and output the i-th decoding feature map until the k-th decoding network outputs the k-th decoding feature map, wherein the k-th decoding feature map includes the first hemi-brain probability map and the second hemi-brain probability map; Wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

9. The method according to claim 8, characterized in that The encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks; the step of extracting features from the encoding feature map output by the previous network based on the i-th encoding network and outputting the i-th encoding feature map comprises: Based on the i-1th downsampling network, downsampling the encoding feature map output by the i-1th encoding network, outputting the i-1th intermediate encoding feature map, inputting the i-1th intermediate encoding feature map into the i-th encoding network, performing feature extraction on the i-1th intermediate encoding feature map based on the i-th encoding network, and outputting the i-th encoding feature map; The method comprises: reconstructing an image of a feature map obtained by fusing a decoding feature map output by a previous network and a (k+1-i)th encoding feature map based on the i-th decoding network, and outputting the i-th decoding feature map. Based on the i-1th upsampling network, the decoding feature map output by the i-1th decoding network is upsampled, and the i-1th intermediate decoding feature map is output. The i-1th intermediate decoding feature map is input into the i-th decoding network. Based on the i-th decoding network, the feature map after the fusion of the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map is reconstructed, and the i-th decoding feature map is output.

10. The method according to claim 1, characterized in that The identifying the contour surface of the brain region from the three-dimensional brain image includes: identifying contour points of the brain region from the three-dimensional brain image; determining a set of contour points of the brain region as a contour surface of the brain region; The step of identifying a first hemisphere outline surface of the first hemisphere region and a second hemisphere outline surface of the second hemisphere region from the three-dimensional brain image comprises: identifying, from the three-dimensional brain image, contour points of the first hemisphere region and contour points of the second hemisphere region; The set of contour points of the first hemisphere region is determined as the first hemisphere contour surface, and the set of contour points of the second hemisphere region is determined as the second hemisphere contour surface.

11. A brain midline identification device, characterized in that: The device comprises: an image acquisition module, configured to acquire a three-dimensional brain image, wherein the three-dimensional brain image includes a brain region; a first recognition module, configured to recognize a contour surface of the brain region from the three-dimensional brain image; an image segmentation module, configured to segment the three-dimensional brain image to obtain a first hemisphere region and a second hemisphere region; a second recognition module, configured to recognize, from the three-dimensional brain image, a first hemisphere outline surface of the first hemisphere region and a second hemisphere outline surface of the second hemisphere region; a brain midline determining module, configured to determine a three-dimensional brain midline of the brain region by taking a difference between a union of the first half-brain contour surface and the second half-brain contour surface and a contour surface of the brain region; a state determination module, configured to determine a state corresponding to the brain region based on the three-dimensional brain image, the state being whether the lesion has broken into the ventricle or the lesion has not broken into the ventricle; a brain midline correction module, configured to, in response to the state that the lesion has not broken into the ventricle, perform image segmentation on the three-dimensional brain image to obtain a lesion region, and determine a target region where the lesion is located based on the lesion region and the three-dimensional brain midline, wherein the target region is the first hemisphere region or the second hemisphere region; The brain midline correction module is further configured to remove a first contour surface located in the target area from a lesion contour surface of the lesion area to obtain a second contour surface, determine two intersection lines between the lesion contour surface and the three-dimensional brain midline, and remove the first brain midline between the two intersection lines from the three-dimensional brain midline to obtain a remaining second brain midline; The brain midline correction module is further used to fit the second brain midline with the second contour surface to obtain a corrected three-dimensional brain midline.

12. The device according to claim 11, characterized in that The image segmentation module is used to: A brain segmentation model is called to perform image segmentation on the three-dimensional brain image to obtain the first hemisphere region and the second hemisphere region.

13. The device according to claim 12, characterized in that The image segmentation module comprises: a first segmentation unit, configured to call the brain segmentation model and perform image segmentation on the three-dimensional brain image to obtain a first brain hemisphere probability map and a second brain hemisphere probability map, wherein the first brain hemisphere probability map is used to represent the probability that each point in the three-dimensional brain image belongs to the first brain hemisphere region, and the second brain hemisphere probability map is used to represent the probability that each point in the three-dimensional brain image belongs to the second brain hemisphere region; a region determining unit, configured to determine, in the first hemisphere probability map, points whose corresponding probabilities are greater than a first probability threshold as points in the first hemisphere region, and to determine a set of points in the first hemisphere region as the first hemisphere region; The region determination unit is further configured to determine, in the second hemisphere probability map, points whose corresponding probabilities are greater than the first probability threshold as points in the second hemisphere region, and to determine a set of points in the second hemisphere region as the second hemisphere region.

14. The device according to claim 13, characterized in that The brain segmentation model includes an encoding sub-model and a decoding sub-model, wherein the encoding sub-model includes k encoding networks and the decoding sub-model includes k decoding networks; the first segmentation unit is used to: Inputting the three-dimensional brain image into a first encoding network, performing feature extraction on the three-dimensional brain image based on the first encoding network, and outputting a first encoding feature map; Based on the i-th encoding network, feature extraction is performed on the encoding feature map output by the previous network, and the i-th encoding feature map is output, until the k-th encoding network outputs the k-th encoding feature map, and the k-th encoding feature map is input into the first decoding network; Reconstructing the k-th encoding feature map based on the first decoding network to obtain a first decoding feature map; Based on the i-th decoding network, reconstruct the feature map after fusing the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map, and output the i-th decoding feature map until the k-th decoding network outputs the k-th decoding feature map, wherein the k-th decoding feature map includes the first hemi-brain probability map and the second hemi-brain probability map; Wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

15. The device according to claim 14, characterized in that The encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks; the first segmentation unit is configured to: Based on the i-1th downsampling network, downsampling the encoding feature map output by the i-1th encoding network, outputting the i-1th intermediate encoding feature map, inputting the i-1th intermediate encoding feature map into the i-th encoding network, performing feature extraction on the i-1th intermediate encoding feature map based on the i-th encoding network, and outputting the i-th encoding feature map; Based on the i-1th upsampling network, the decoding feature map output by the i-1th decoding network is upsampled, and the i-1th intermediate decoding feature map is output. The i-1th intermediate decoding feature map is input into the i-th decoding network. Based on the i-th decoding network, the feature map after the fusion of the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map is reconstructed, and the i-th decoding feature map is output.

16. The device according to claim 13, characterized in that The training process of the brain segmentation model includes: Acquire first sample data, the first sample data including a sample three-dimensional brain image and a first sample segmented image, wherein the first sample segmented image is annotated with a first hemisphere region and a second hemisphere region in the sample three-dimensional brain image; Invoking the brain segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain a first half-brain probability map and a second half-brain probability map, wherein the first half-brain probability map is used to represent the probability that each point in the sample three-dimensional brain image belongs to the first half-brain region, and the second half-brain probability map is used to represent the probability that each point in the sample three-dimensional brain image belongs to the second half-brain region; The brain segmentation model is trained based on the first brain hemisphere probability map, the second brain hemisphere probability map, and the first sample segmentation image.

17. The device according to claim 16, characterized in that The training of the brain segmentation model based on the first brain hemisphere probability map, the second brain hemisphere probability map, and the first sample segmentation image includes: determining, based on the first sample segmented image, at least one point belonging to the first hemisphere region and at least one point belonging to the second hemisphere region; Determining, based on the first hemisphere probability map, a probability that the at least one point belonging to the first hemisphere region belongs to the first hemisphere region, and determining, based on the second hemisphere probability map, a probability that the at least one point belonging to the second hemisphere region belongs to the second hemisphere region; determining a loss value of the brain segmentation model based on a probability that the at least one point belonging to the first hemisphere region belongs to the first hemisphere region and a probability that the at least one point belonging to the second hemisphere region belongs to the second hemisphere region; The brain segmentation model is trained based on the loss value.

18. The device according to claim 16, characterized in that The brain segmentation model includes an encoding sub-model and a decoding sub-model, wherein the encoding sub-model includes k encoding networks and the decoding sub-model includes k decoding networks; The calling of the brain segmentation model to perform image segmentation on the sample three-dimensional brain image to obtain a first half brain probability map and a second half brain probability map includes: Inputting the sample three-dimensional brain image into a first encoding network, performing feature extraction on the sample three-dimensional brain image based on the first encoding network, and outputting a first encoding feature map; Based on the i-th encoding network, feature extraction is performed on the encoding feature map output by the previous network, and the i-th encoding feature map is output, until the k-th encoding network outputs the k-th encoding feature map, and the k-th encoding feature map is input into the first decoding network; Reconstructing the k-th encoding feature map based on the first decoding network to obtain a first decoding feature map; Based on the i-th decoding network, reconstruct the feature map after fusing the decoding feature map output by the previous network and the (k+1-i)-th encoding feature map, and output the i-th decoding feature map until the k-th decoding network outputs the k-th decoding feature map, wherein the k-th decoding feature map includes the first hemi-brain probability map and the second hemi-brain probability map; Wherein, k is a positive integer greater than 1, and i is any positive integer greater than 1 and not greater than k.

19. The device according to claim 18, characterized in that The encoding sub-model further includes a downsampling network between every two encoding networks, and the decoding sub-model further includes an upsampling network between every two decoding networks; the step of extracting features from the encoding feature map output by the previous network based on the i-th encoding network and outputting the i-th encoding feature map comprises: Based on the i-1th downsampling network, downsampling the encoding feature map output by the i-1th encoding network, outputting the i-1th intermediate encoding feature map, inputting the i-1th intermediate encoding feature map into the i-th encoding network, performing feature extraction on the i-1th intermediate encoding feature map based on the i-th encoding network, and outputting the i-th encoding feature map; The method comprises: reconstructing an image of a feature map obtained by fusing a decoding feature map output by a previous network and a (k+1-i)th encoding feature map based on the i-th decoding network, and outputting the i-th decoding feature map. Based on the i-1th upsampling network, the decoding feature map output by the i-1th decoding network is upsampled, and the i-1th intermediate decoding feature map is output. The i-1th intermediate decoding feature map is input into the i-th decoding network. Based on the i-th decoding network, the feature map after the fusion of the i-1th intermediate decoding feature map and the (k+1-i)th encoding feature map is reconstructed, and the i-th decoding feature map is output.

20. The device according to claim 11, characterized in that The first identification module is configured to: identifying contour points of the brain region from the three-dimensional brain image; determining a set of contour points of the brain region as a contour surface of the brain region; The second identification module is configured to: identifying, from the three-dimensional brain image, contour points of the first hemisphere region and contour points of the second hemisphere region; The set of contour points of the first hemisphere region is determined as the first hemisphere contour surface, and the set of contour points of the second hemisphere region is determined as the second hemisphere contour surface.

21. A computer device, characterized in that: The computer device includes a processor and a memory, wherein at least one computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the operations performed by the brain midline recognition method according to any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that The storage medium stores at least one computer program, which is loaded and executed by a processor to implement the operations performed by the brain midline recognition method according to any one of claims 1 to 10.

23. A computer program product, characterized in that The computer program product includes a computer program, which is stored in a computer-readable storage medium. The processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs the operations performed by the brain midline identification method according to any one of claims 1 to 10.

Images