A method, system and device for automatically delineating and detecting the offset of the brain midline

Abstract

The present invention discloses a method, system, device and storage medium for automatically outlining and detecting the offset of a brain midline. The method comprises: obtaining an original brain CT image and then calibrating it to obtain a calibrated brain CT image; processing the calibrated brain CT image using a brain midline extraction model to obtain a feature image containing only a predicted brain midline; superimposing the feature image and the calibrated brain CT image to obtain a target brain CT image that automatically outlines the predicted brain midline; in the feature image, automatically generating an ideal brain midline according to the two endpoints of the predicted brain midline, and determining the offset of the predicted brain midline relative to the ideal brain midline. The present invention can accurately and reliably extract the slender and complex tubular structure feature of the brain midline contained in the brain CT image by introducing two different tubular convolutional layers in the encoder structure and the decoder structure contained in the brain midline extraction model.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, system, device and storage medium for automatically delineating and detecting the offset of a brain midline. Background Art

[0002] In the field of medical image processing, accurate identification of the brain midline and calculation of its offset are of great significance for the diagnosis of craniocerebral injury, cerebral hemorrhage and other diseases. Traditional brain CT (Computed Tomography) image analysis often requires doctors to manually outline the brain midline, which is not only time-consuming and laborious, but also has subjective errors and individual differences. To this end, some scholars have proposed using traditional convolutional neural networks to identify and extract the brain midline from brain CT images. However, since the brain midline has a slender tubular feature, the convolution kernel of a fixed shape and size used by the convolutional neural network is often difficult to fully capture this tubular feature in the brain CT image, resulting in the inability to accurately and reliably outline the brain midline, and thus the inability to further accurately calculate the offset of the brain midline. Summary of the invention

[0003] The present invention provides a method, system, device and storage medium for automatically delineating and detecting the offset of the brain midline, so as to solve one or more technical problems existing in the prior art and at least provide a beneficial choice or create conditions.

[0004] In a first aspect, a method for automatically delineating a brain midline and detecting an offset is provided, the method comprising:

[0005] Obtaining original brain CT images;

[0006] Calibrate the original brain CT image to obtain a calibrated brain CT image;

[0007] Processing the calibrated brain CT image using the trained brain midline extraction model to obtain a feature image containing only the predicted brain midline;

[0008] superimposing the characteristic image and the calibrated brain CT image to obtain a target brain CT image that automatically outlines the predicted brain midline;

[0009] In the characteristic image, an ideal brain midline is automatically generated according to two end points of the predicted brain midline, and then an offset of the predicted brain midline relative to the ideal brain midline is determined.

[0010] Furthermore, calibrating the original brain CT image to obtain a calibrated brain CT image includes:

[0011] Preprocessing the original brain CT image to obtain a brain CT image to be tested;

[0012] Among them, the preprocessing methods include binarization and morphological closing operation;

[0013] Extracting all boundary points on the maximum closed contour line from the brain CT image to be tested, and then performing ellipse fitting on all the boundary points to obtain an ellipse curve equation;

[0014] Determining the rotation angle of the fitted ellipse according to the parameter information contained in the elliptic curve equation;

[0015] Determining whether the rotation angle is zero;

[0016] If yes, the original brain CT image is directly used as the calibration brain CT image;

[0017] If not, determining the rotation matrix of the ellipse according to the parameter information and the rotation angle;

[0018] The original brain CT image is multiplied by the rotation matrix to obtain a calibrated brain CT image.

[0019] Furthermore, the brain midline extraction model includes an encoder and a decoder, the encoder includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module, a first downsampling layer, a second downsampling layer and a third downsampling layer, and the decoder includes a first feature reconstruction module, a second feature reconstruction module, a third feature reconstruction module, a first upsampling layer, a second upsampling layer and a third upsampling layer;

[0020] The output end of the first feature extraction module is connected to the input end of the second feature extraction module and the input end of the third feature reconstruction module respectively through the first down-sampling layer, the output end of the second feature extraction module is connected to the input end of the third feature extraction module and the input end of the second feature reconstruction module respectively through the second down-sampling layer, the output end of the third feature extraction module is connected to the input end of the fourth feature extraction module and the input end of the first feature reconstruction module respectively through the third down-sampling layer, the output end of the fourth feature extraction module is connected to the input end of the first feature reconstruction module through the first up-sampling layer, the output end of the first feature reconstruction module is connected to the input end of the second feature reconstruction module through the second up-sampling layer, and the output end of the second feature reconstruction module is connected to the input end of the third feature reconstruction module through the third up-sampling layer;

[0021] Each feature extraction module includes three first convolutional layers and a first splicing layer, wherein the first input image is processed by the three first convolutional layers to obtain three corresponding first feature maps, and the three first feature maps are spliced ​​in the channel dimension direction by the first splicing layer to obtain a first output image; wherein the three first convolutional layers include a first 3×3 convolutional layer, a first 9×1 tubular convolutional layer, and a first 16×1 tubular convolutional layer;

[0022] Each feature reconstruction module includes three second convolutional layers, a second splicing layer and a third splicing layer. The second splicing layer is used to splice the two second input images in the channel dimension direction to obtain a second intermediate image. The three second convolutional layers are used to process the second intermediate image to obtain three corresponding second feature maps. The third splicing layer is used to splice the three second feature maps in the channel dimension direction to obtain a second output image; wherein the three second convolutional layers include a second 3×3 convolutional layer, a second 9×1 tubular convolutional layer and a second 16×1 tubular convolutional layer.

[0023] Furthermore, the processing of each 9×1 tubular convolutional layer includes:

[0024] The third input image is processed using a 9×1 convolution kernel to obtain an offset of each pixel in the third input image and record the offset as first pixel offset information;

[0025] Using the first pixel offset information, fine-tuning the position of each pixel in the third input image to obtain a third intermediate image;

[0026] Perform bilinear interpolation on the third intermediate image to obtain a third output image.

[0027] Furthermore, the processing of each 16×1 tubular convolutional layer includes:

[0028] Processing the fourth input image with a 16×1 convolution kernel to obtain an offset of each pixel in the fourth input image and record the offset as second pixel offset information;

[0029] fine-adjusting the position of each pixel in the fourth input image by using the second pixel offset information to obtain a fourth intermediate image;

[0030] Perform bilinear interpolation on the fourth intermediate image to obtain a fourth output image.

[0031] Further, determining the offset of the predicted brain midline relative to the ideal brain midline includes:

[0032] In the feature image, all pixel points on the predicted brain center line are extracted to form a predicted point set, and all pixel points on the ideal brain center line are extracted to form an ideal point set;

[0033] For each pixel in the prediction point set, the pixel is recorded as a prediction pixel and a circular neighborhood is created with the pixel as the center, and then all the pixels falling in the circular neighborhood are screened out from the ideal point set, and the number of all the pixels is used as the density value of the prediction pixel;

[0034] A pixel point with the smallest density value is selected from the predicted point set and recorded as a target pixel point, and then the vertical distance between the target pixel point and the ideal brain midline is used as an offset.

[0035] Furthermore, the method further includes: visualizing the target brain CT image and the offset.

[0036] In a second aspect, a system for automatically delineating and detecting the offset of a brain midline is provided, the system comprising:

[0037] The first module is used to obtain the original brain CT image;

[0038] The second module is used to calibrate the original brain CT image to obtain a calibrated brain CT image;

[0039] A third module is used to process the calibrated brain CT image using the trained brain midline extraction model to obtain a feature image containing only the predicted brain midline;

[0040] A fourth module is used to superimpose the characteristic image and the calibrated brain CT image to obtain a target brain CT image that automatically outlines the predicted brain midline;

[0041] The fifth module is used to automatically generate an ideal brain midline in the feature image according to the two endpoints of the predicted brain midline, and then determine the offset of the predicted brain midline relative to the ideal brain midline.

[0042] In a third aspect, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method for automatic delineation and offset detection of the brain midline as described in the first aspect.

[0043] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the method for automatically delineating the brain midline and detecting the offset as described in the first aspect is implemented.

[0044] The present invention has at least the following beneficial effects: by introducing two different tubular convolutional layers in both the encoder structure and the decoder structure contained in the brain midline extraction model, accurate and reliable extraction of the slender and complex tubular structure feature of the brain midline contained in the brain CT image can be achieved; by introducing the ellipse fitting principle based on the least squares method to calibrate the direction of the brain CT image, an accurate basis can be provided for the subsequent brain midline delineation task; the overall process of image calibration, brain midline delineation in the image, and brain midline offset analysis in the image proposed by the present invention is fully automated, reducing human intervention and improving image processing efficiency and detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0046] Figure 1 is a flow chart of a method for automatically delineating the brain midline and detecting the offset in an embodiment of the present invention;

[0047] Figure 2 is a structural schematic diagram of a brain midline extraction model in an embodiment of the present invention;

[0048] Figure 3 is a schematic diagram of the structure of any feature extraction module in an embodiment of the present invention;

[0049] Figure 4 is a schematic diagram of the structure of any feature reconstruction module in an embodiment of the present invention;

[0050] Figure 5 Schematic diagram of the composition of the brain midline automatic delineation and offset detection system in an embodiment of the present invention;

[0051] Figure 6 It is a schematic diagram of the hardware structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0053] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order from the module division in the system or the order in the flow chart. The terms "first", "second", "third", "fourth", etc. in the specification of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, such as processes, methods, systems, products or devices that include a series of steps or units, which are not necessarily limited to those steps or units clearly listed, but may include other steps or units inherent to these processes, methods, products or devices that are not clearly listed.

[0054] Please refer to Figure 1 , Figure 1 : is a flow chart of a method for automatically delineating a brain midline and detecting an offset provided by an embodiment of the present invention, the method comprising the following steps:

[0055] Step S110, obtaining an original brain CT image;

[0056] Step S120, calibrating the original brain CT image to obtain a calibrated brain CT image;

[0057] Step S130, using the trained brain midline extraction model to process the calibrated brain CT image to obtain a feature image containing only the predicted brain midline;

[0058] Step S140, superimposing the characteristic image and the calibrated brain CT image to obtain a target brain CT image that automatically outlines the predicted brain midline;

[0059] Step S150: In the feature image, an ideal brain midline is automatically generated according to two endpoints of the predicted brain midline, and then an offset of the predicted brain midline relative to the ideal brain midline is determined.

[0060] In some embodiments of the present invention, the implementation process of step S120 includes but is not limited to the following:

[0061] Step S121, performing a preprocessing operation on the original brain CT image to obtain a brain CT image to be tested; wherein the preprocessing operation mainly includes binarization and morphological closing operation;

[0062] Step S122, screening out the maximum closed contour line in the brain CT image to be tested and acquiring all boundary points thereon;

[0063] Step S123, performing an ellipse fitting operation on all the boundary points to obtain an elliptic curve equation;

[0064] Step S124: determining the rotation angle of the fitted ellipse based on the parameter information provided by the elliptic curve equation;

[0065] Step S125, determining whether the rotation angle is zero; if so, it means that the ellipse is not deflected relative to the horizontal plane, and the original brain CT image can be directly defined as the calibrated brain CT image; if not, it means that the ellipse is deflected relative to the horizontal plane, and step S126 is executed;

[0066] Step S126: determining a rotation matrix of the ellipse based on the rotation angle and the parameter information;

[0067] Step S127: multiply the original brain CT image and the rotation matrix to obtain a calibrated brain CT image.

[0068] More specifically, the implementation process of step S121 includes but is not limited to the following:

[0069] Step S121.1, performing a binarization operation on the original brain CT image to obtain an intermediate brain CT image. The mathematical expression used in this embodiment is:

[0070]

[0071] Step S121.2, performing a morphological closing operation on the intermediate brain CT image to obtain the brain CT image to be tested. The mathematical expression used in this embodiment is:

[0072]

[0073] Wherein, I0(x,y) is the pixel value at the position (x,y) in the original brain CT image, T is a preset binarization threshold, B(x,y) is the pixel value at the position (x,y) in the intermediate brain CT image, 1 refers to the target area, 0 refers to the background area, K is a preset structural element, I(x,y) is the pixel value at the position (x,y) in the brain CT image to be tested, Refers to the expansion operation, Refers to the corrosion operation, by performing the morphological closing operation on the intermediate brain CT image, the small holes in the target area can be filled, the boundary of the target area can be smoothed, and the isolated pixels and the hole areas in the image can be eliminated.

[0074] More specifically, the implementation process of the above step S122 includes but is not limited to the following:

[0075] Step S122.1, extracting all closed areas from the brain CT image to be tested, wherein all pixel values ​​in each closed area are 1;

[0076] Step S122.2, calculating the area of ​​each closed region, and then selecting the closed region with the largest area from all the closed regions and recording it as the largest closed region;

[0077] Step S122.3: Obtain the maximum closed contour line on the maximum closed area, and then obtain all pixel points on the maximum closed contour line and define them as all boundary points.

[0078] More specifically, the implementation process of the above step S123 includes but is not limited to the following:

[0079] Step S123.1, set the general expression of the elliptic curve equation to:

[0080] x 2 +Axy+By 2 +Cx+Dy+E=0;

[0081] According to the principle of least squares method, the objective function required for fitting is constructed as follows:

[0082]

[0083] In order to minimize the objective function F, it is necessary to make the partial derivatives zero, that is:

[0084]

[0085] Combining the expression of the objective function F and the partial derivative results, the following equation can be obtained:

[0086]

[0087]

[0088]

[0089] Wherein, A, B, C, D, E are five parameter information provided by the elliptic curve equation, N is the number of all boundary points, (x i ,y i ) is the coordinate position of the i-th boundary point among all the boundary points in the brain CT image to be tested, M1 and M2 are both reference matrices, T represents the matrix transposition symbol, and -1 represents the matrix inversion symbol;

[0090] Step S123.2, obtaining the coordinate position of each boundary point of all the boundary points in the brain CT image to be tested, and then substituting the coordinate position into the above equations for solving, to obtain the specific values ​​of the five parameter information;

[0091] Step S123.3: Substitute the specific values ​​of the five parameter information into the general expression of the elliptic curve equation to obtain the final required elliptic curve equation.

[0092] In the above step S124, the solution formula for the rotation angle of the ellipse is:

[0093]

[0094] Where θ is the rotation angle.

[0095] In the above step S126, the solution formula for the rotation matrix of the ellipse is:

[0096]

[0097]

[0098] Where R is the rotation matrix, and φ is the angle between the vertical direction of the major axis of the ellipse and the horizontal plane.

[0099] In some embodiments of the present invention, see Figure 2 As shown, the brain midline extraction model mentioned in the above step S130 includes an encoder and a decoder, the encoder includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module, a first down-sampling layer, a second down-sampling layer and a third down-sampling layer, wherein each down-sampling layer preferably adopts a maximum pooling layer, and the decoder includes a first feature reconstruction module, a second feature reconstruction module, a third feature reconstruction module, a first up-sampling layer, a second up-sampling layer and a third up-sampling layer;

[0100] The output end of the first feature extraction module is connected to the input end of the second feature extraction module and the input end of the third feature reconstruction module respectively through the first down-sampling layer, the output end of the second feature extraction module is connected to the input end of the third feature extraction module and the input end of the second feature reconstruction module respectively through the second down-sampling layer, the output end of the third feature extraction module is connected to the input end of the fourth feature extraction module and the input end of the first feature reconstruction module respectively through the third down-sampling layer, the output end of the fourth feature extraction module is connected to the input end of the first feature reconstruction module through the first up-sampling layer, the output end of the first feature reconstruction module is connected to the input end of the second feature reconstruction module through the second up-sampling layer, and the output end of the second feature reconstruction module is connected to the input end of the third feature reconstruction module through the third up-sampling layer.

[0101] In actual application, the input calibrated brain CT image is processed by the first feature extraction module to obtain a first low-level feature image, the first low-level feature image is processed by the first downsampling layer to obtain a first final low-level feature image, the first final low-level feature image is processed by the second feature extraction module to obtain a second low-level feature image, the second low-level feature image is processed by the second downsampling layer to obtain a second final low-level feature image, the second final low-level feature image is processed by the third feature extraction module to obtain a third low-level feature image, the third low-level feature image is processed by the third downsampling layer to obtain a third final low-level feature image, and the third final low-level feature image is processed by the fourth feature extraction module to obtain a fourth low-level feature image;

[0102] The fourth low-level feature image is processed by the first upsampling layer to obtain a fourth final low-level feature image, the third final low-level feature image and the fourth final low-level feature image are fused by the first feature reconstruction module to obtain a first high-level feature image, the first high-level feature image is processed by the second upsampling layer to obtain a first final high-level feature image, the second final low-level feature image and the first final high-level feature image are fused by the second feature reconstruction module to obtain a second high-level feature image, the second high-level feature image is processed by the third upsampling layer to obtain a second final high-level feature image, and the first final low-level feature image and the second final high-level feature image are fused by the third feature reconstruction module to obtain the final required feature image output.

[0103] More specifically, in the encoder, see Figure 3As shown, any one of the feature extraction modules includes a first splicing layer and three first convolutional layers, wherein the three first convolutional layers include a first 16×1 tubular convolutional layer, a first 9×1 tubular convolutional layer, and a first 3×3 convolutional layer; the input end of the first splicing layer is respectively connected to the output end of the first 16×1 tubular convolutional layer, the output end of the first 9×1 tubular convolutional layer, and the output end of the first 3×3 convolutional layer.

[0104] In actual application, when the feature extraction module receives a first input image, the first input image is processed by the first 16×1 tubular convolution layer to obtain a first first feature map, the first input image is processed by the first 9×1 tubular convolution layer to obtain a second first feature map, the first input image is processed by the first 3×3 convolution layer to obtain a third first feature map, and the three first feature maps are spliced ​​in the channel dimension direction by the first splicing layer to obtain a first output image.

[0105] More specifically, in the decoder, see Figure 4 As shown, any one of the feature reconstruction modules includes a second splicing layer, a third splicing layer and three second convolutional layers, wherein the three second convolutional layers include a second 16×1 tubular convolutional layer, a second 9×1 tubular convolutional layer and a second 3×3 convolutional layer; the output end of the second splicing layer is respectively connected to the input end of the second 16×1 tubular convolutional layer, the input end of the second 9×1 tubular convolutional layer and the input end of the second 3×3 convolutional layer, and the input end of the third splicing layer is respectively connected to the output end of the second 16×1 tubular convolutional layer, the output end of the second 9×1 tubular convolutional layer and the output end of the second 3×3 convolutional layer.

[0106] In actual application, when the feature reconstruction module receives two second input images, the two second input images are spliced ​​in the channel dimension direction through the second splicing layer to obtain a second intermediate image, the second intermediate image is processed by the second 16×1 tubular convolution layer to obtain a first second feature map, the second intermediate image is processed by the second 9×1 tubular convolution layer to obtain a second second feature map, the second intermediate image is processed by the second 3×3 convolution layer to obtain a third second feature map, and the three second feature maps are spliced ​​in the channel dimension direction through the third splicing layer to obtain a second output image.

[0107] It should be noted that any tubular convolution layer included in the encoder and the decoder can actually be understood as a deformable convolution layer.

[0108] More specifically, for any of the 9×1 tubular convolutional layers mentioned above, when the 9×1 tubular convolutional layer receives the third input image, at least the following three steps need to be performed:

[0109] A1. Perform a convolution operation on the third input image through a 9×1 convolution kernel to obtain an offset of each pixel contained in the third input image (mainly including an X-axis offset and a Y-axis offset), and record it as first pixel offset information. The mathematical expression used in this implementation process is:

[0110] P1=Conv 9×1 (F1);

[0111] A2. Based on the first pixel offset information, a position fine-tuning operation is performed on each pixel included in the third input image to obtain a third intermediate image. The mathematical expression used in this implementation process is:

[0112]

[0113] A3. Perform a bilinear interpolation operation on the third intermediate image to obtain a third output image. The mathematical expression used in this implementation process is:

[0114]

[0115] Among them, O1 is the first pixel offset information, which can also be called the predicted first offset tensor, Conv 9×1 refers to the 9×1 convolution kernel, F1 is the third input image, (X 1_ij ,Y 1_ij ) is the coordinate position of the pixel contained in the third input image, i represents the index of the pixel in the horizontal (width) direction of the image, j represents the index of the pixel in the vertical (height) direction of the image, (O 1_x,ij ,O 1_y,ij ) is the pixel (X) contained in the first pixel offset information 1_ij ,Y 1_ij ), (X1′ _ij ,Y1′ _ij ) is the coordinate position of the pixel contained in the third intermediate image, which is (X 1_ij ,Y 1_ij ), S is the preset expansion range, which is a scalar value used to control the degree of influence of the offset, F1′(i,j) is the pixel value at the integer coordinate position (i,j) contained in the third intermediate image, F1″(x,y) is the pixel value at the interpolation position (x,y) contained in the third output image, ∑ i,jIt indicates that the pixel values ​​at the four integer coordinate positions closest to the interpolation position (x, y) contained in the third intermediate image are selected to complete the interpolation operation.

[0116] More specifically, for any of the 16×1 tubular convolutional layers mentioned above, when the 16×1 tubular convolutional layer receives the fourth input image, at least the following three steps need to be performed:

[0117] B1. Perform a convolution operation on the fourth input image through a 16×1 convolution kernel to obtain an offset of each pixel contained in the fourth input image (mainly including an X-axis offset and a Y-axis offset), and record it as second pixel offset information. The mathematical expression used in this implementation process is:

[0118] O2=Conv 16×1 (F2);

[0119] B2. Based on the second pixel offset information, a position fine-tuning operation is performed on each pixel included in the fourth input image to obtain a fourth intermediate image. The mathematical expression used in this implementation process is:

[0120]

[0121] B3. Perform a bilinear interpolation operation on the fourth intermediate image to obtain a fourth output image. The mathematical expression used in this implementation process is:

[0122]

[0123] Among them, O2 is the second pixel offset information, which can also be called the predicted second offset tensor, Conv 16×1 refers to the 16×1 convolution kernel, F2 is the fourth input image, (X 2_ij ,Y 2_ij ) is the coordinate position of the pixel contained in the fourth input image, i represents the index of the pixel in the horizontal (width) direction of the image, j represents the index of the pixel in the vertical (height) direction of the image, (O 2_x,ij ,O 2_y,ij ) is the pixel (X) contained in the second pixel offset information 2_ij ,Y 2_ij ) offset, (X2′ _ij ,Y2′ _ij ) is the coordinate position of the pixel contained in the fourth intermediate image, which is (X 2_ij ,Y 2_ij), S is the preset expansion range, which is a scalar value used to control the degree of influence of the offset, F2′(i,j) is the pixel value at the integer coordinate position (i,j) contained in the fourth intermediate image, F2″(x,y) is the pixel value at the interpolation position (x,y) contained in the fourth output image, ∑ i,j It indicates that the pixel values ​​at the four integer coordinate positions closest to the interpolation position (x, y) contained in the fourth intermediate image are selected to complete the interpolation operation.

[0124] In the above step A1 and the above step B1, the purpose of offset prediction is to learn the characteristics of the input image and the potential deformation of the brain midline structure, and to limit the offset of each pixel to fall within the range of [-1,1] to ensure that the features extracted by the convolution operation are tubular.

[0125] In the above step A3 and the above step B3, after each pixel contained in the input image is dynamically adjusted to adapt to the corresponding offset, the new coordinate position of the pixel may not fall completely on the integer coordinate position of the original grid, that is, the new coordinate position of the pixel may be a decimal, but in general, the pixel value can only be defined at the integer coordinate position, so it is necessary to use a bilinear interpolation algorithm to solve the problem that the new coordinate position of the pixel may be a decimal.

[0126] It should be noted that the reason for choosing to use 9×1 convolution kernels and 16×1 convolution kernels in the brain midline extraction model is that the original brain CT image is calibrated in the Y-axis direction in the above step S120; conversely, if the original brain CT image is adjusted to be calibrated in the X-axis direction in the above step S120, the brain midline extraction model is adjusted to use 1×9 convolution kernels and 1×16 convolution kernels.

[0127] In some embodiments of the present invention, the training process of the brain midline extraction model is described, including but not limited to the following steps:

[0128] (1) Collecting a number of brain CT images from a medical system or a related website, wherein the plurality of brain CT images include brain CT images with cerebral hemorrhage and brain CT images without abnormal phenomena, and then having experienced medical staff manually outline the brain midline for each collected brain CT image, thereby obtaining an original data set;

[0129] (2) According to the implementation principle of the above step S120, each brain CT image contained in the original data set is calibrated, and then a data enhancement operation is performed on each calibrated brain CT image to obtain a training data set; wherein the data enhancement method includes normalization, random left-right flipping and random cropping, etc., to enhance the diversity and generalization ability of the data;

[0130] (3) Determine the final loss function required for training, where the final loss function is obtained by weighted summation of a BCE (Binary Cross-Entropy) loss function and a Dice loss function;

[0131] (4) The constructed brain midline extraction model is trained using the training data set, and the final loss function is used as a reference factor during the training process. The network parameters of the brain midline extraction model are updated through a back propagation algorithm, so that the brain midline extraction model can gradually and better learn the brain midline features in the input brain CT image.

[0132] It should be noted that, in the above step S140, the calibration brain CT image and the feature image are of the same size, and the target brain CT image can be understood as the predicted brain midline in the feature image being aligned and marked on the calibration brain CT image.

[0133] In some embodiments of the present invention, the implementation process of step S150 includes but is not limited to the following:

[0134] Step S151, automatically connecting the two endpoints of the predicted brain midline contained in the feature image to form a straight line segment, and then defining the straight line segment as the ideal brain midline;

[0135] Step S152, extracting all pixel points on the predicted brain midline from the feature image to form a predicted point set;

[0136] Step S153, extracting all pixel points on the ideal brain centerline from the feature image to form an ideal point set;

[0137] Step S154: for each pixel point included in the prediction point set, define the pixel point as a prediction pixel point, create a circular neighborhood with a predetermined radius with the prediction pixel point as the center, then filter the ideal point set to extract all the pixel points in the circular neighborhood, count the number of all the pixel points and define it as the density value of the prediction pixel point;

[0138] It should be noted that the smaller the density value of the predicted pixel point is, the greater the degree to which the predicted pixel point deviates from the ideal brain midline is; conversely, the larger the density value of the predicted pixel point is, the smaller the degree to which the predicted pixel point deviates from the ideal brain midline is;

[0139] Step S155, screening the predicted point set to extract the pixel with the smallest density value and defining it as the target pixel, then calculating the vertical distance between the target pixel and the ideal brain center line and defining it as the final required offset.

[0140] As an optional implementation scheme, the implementation process of the above step S150 may also include but is not limited to the following:

[0141] Step S150.1, automatically connecting the two endpoints of the predicted brain midline contained in the feature image to form a straight line segment, and then defining the straight line segment as the ideal brain midline;

[0142] Step S150.2, extracting all pixel points on the predicted brain midline from the feature image to form a first predicted point set;

[0143] Step S150.3, extracting all pixel points on the ideal brain midline from the feature image to form a first ideal point set;

[0144] Step S150.4: for each pixel point included in the first prediction point set, define the pixel point as a first prediction pixel point, calculate the distance values ​​between all the pixels included in the first ideal point set and the first prediction pixel point, and then extract the minimum value from all the calculated distance values ​​and associate it with the first prediction pixel point;

[0145] Step S150.5: Screen all the distance values ​​associated and bound in the first prediction point set to extract the maximum value and define it as the final required offset.

[0146] As a preferred embodiment, after executing the above step S150, a visualization operation can also be performed on the offset and the target brain CT image; of course, in order to facilitate doctors or researchers to more intuitively observe the automatic delineation of the predicted brain midline, the predicted brain midline presented in the target brain CT image can be set to a colored line to enhance the visualization effect.

[0147] In an embodiment of the present invention, by introducing two different tubular convolutional layers in the encoder structure and the decoder structure contained in the brain midline extraction model, the slender and complex tubular structure feature of the brain midline contained in the brain CT image can be accurately and reliably extracted; by introducing the ellipse fitting principle based on the least squares method to calibrate the direction of the brain CT image, an accurate basis can be provided for the subsequent brain midline delineation task; the overall process of image calibration, brain midline delineation in the image, and brain midline offset analysis in the image proposed by the present invention is fully automated, reducing human intervention and improving image processing efficiency and detection accuracy.

[0148] Please refer to Figure 5 , Figure 5 : is a schematic diagram of a brain midline automatic delineation and offset detection system provided by an embodiment of the present invention, the system comprising:

[0149] The first module 210 is used to obtain an original brain CT image;

[0150] The second module 220 is used to perform a calibration operation on the original brain CT image to obtain a calibrated brain CT image;

[0151] The third module 230 is used to input the calibrated brain CT image into the trained brain midline extraction model for processing to obtain a feature image containing only the predicted brain midline;

[0152] A fourth module 240 is used to perform a superposition operation on the calibrated brain CT image and the characteristic image to obtain a target brain CT image that automatically outlines the predicted brain midline;

[0153] The fifth module 250 is used to automatically generate an ideal brain midline in the feature image based on the two endpoints of the predicted brain midline, and then determine the offset of the predicted brain midline relative to the ideal brain midline.

[0154] The contents of the above method embodiments are all applicable to the present system embodiments. The functions implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are the same as those of the above method embodiments, which will not be repeated here.

[0155] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, a method for automatically delineating and detecting the offset of a brain centerline in the above embodiment is implemented. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card. In other words, the storage device includes any medium that stores or transmits information in a readable form by a device (such as a computer, a mobile phone, etc.), which can be a read-only memory, a disk or an optical disk, etc.

[0156] also, Figure 6 3 is a schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention, wherein the computer device includes a processor 320, a memory 330, an input unit 340, a display unit 350 and other devices. Those skilled in the art can understand that Figure 6 The device structure device shown does not constitute a limitation on all devices, and may include more or fewer components than shown, or combine certain components. The memory 330 can be used to store the computer program 310 and various functional modules, and the processor 320 runs the computer program 310 stored in the memory 330, thereby executing various functional applications and data processing of the device. The memory can be an internal memory or an external memory, or include an internal memory and an external memory. The internal memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, or a random access memory. The external memory may include a hard disk, a floppy disk, a U disk, a magnetic tape, etc. The memory 330 disclosed in the embodiment of the present invention includes but is not limited to the above-mentioned types of memory. The memory 330 disclosed in the embodiment of the present invention is only an example and not a limitation.

[0157] The input unit 340 is used to receive the input of the signal and the keyword input by the user. The input unit 340 may include a touch panel and other input devices. The touch panel can collect the user's touch operation on or near it (such as the user's operation on or near the touch panel using any suitable object or accessory such as a finger, a stylus, etc.), and drive the corresponding connection device according to a pre-set program; other input devices may include but are not limited to one or more of a physical keyboard, a function key (such as a playback control key, a switch key, etc.), a trackball, a mouse, a joystick, etc. The display unit 350 can be used to display the information input by the user or the information provided to the user and various menus of the terminal device. The display unit 350 can be in the form of a liquid crystal display, an organic light emitting diode, etc. The processor 320 is the control center of the terminal device, which uses various interfaces and lines to connect various parts of the entire device, and executes various functions and processes data by running or executing software programs and / or modules stored in the memory 330, and calling data stored in the memory 330.

[0158] As an embodiment, the computer device includes a processor 320, a memory 330 and a computer program 310, wherein the computer program 310 is stored in the memory 330 and is configured to be executed by the processor 320, and the computer program 310 is configured to execute a method for automatic delineation and offset detection of a brain midline in the above-mentioned embodiment.

[0159] Although the description of the present application has been quite detailed and specifically describes several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad possible interpretation of these claims by reference to the attached claims, taking into account the prior art, so as to effectively cover the intended scope of the present application. In addition, the above description of the present application is based on the embodiments foreseeable by the inventor, and its purpose is to provide a useful description, and those non-substantial changes to the present application that have not yet been foreseen may still represent equivalent changes to the present application.

Claims

1. A method for automatically delineating and detecting the offset of the brain midline, characterized in that: The method comprises: Obtain original brain CT images; Calibrate the original brain CT image to obtain a calibrated brain CT image; Processing the calibrated brain CT image using the trained brain midline extraction model to obtain a feature image containing only the predicted brain midline; superimposing the characteristic image and the calibrated brain CT image to obtain a target brain CT image that automatically outlines the predicted brain midline; In the characteristic image, an ideal brain midline is automatically generated according to two endpoints of the predicted brain midline, and then an offset of the predicted brain midline relative to the ideal brain midline is determined; The brain midline extraction model includes an encoder and a decoder, the encoder includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module, a first downsampling layer, a second downsampling layer and a third downsampling layer, and the decoder includes a first feature reconstruction module, a second feature reconstruction module, a third feature reconstruction module, a first upsampling layer, a second upsampling layer and a third upsampling layer; The output end of the first feature extraction module is connected to the input end of the second feature extraction module and the input end of the third feature reconstruction module respectively through the first down-sampling layer, the output end of the second feature extraction module is connected to the input end of the third feature extraction module and the input end of the second feature reconstruction module respectively through the second down-sampling layer, the output end of the third feature extraction module is connected to the input end of the fourth feature extraction module and the input end of the first feature reconstruction module respectively through the third down-sampling layer, the output end of the fourth feature extraction module is connected to the input end of the first feature reconstruction module through the first up-sampling layer, the output end of the first feature reconstruction module is connected to the input end of the second feature reconstruction module through the second up-sampling layer, and the output end of the second feature reconstruction module is connected to the input end of the third feature reconstruction module through the third up-sampling layer; Each feature extraction module includes three first convolutional layers and a first splicing layer, wherein the first input image is processed by the three first convolutional layers to obtain three corresponding first feature maps, and the three first feature maps are spliced ​​in the channel dimension direction by the first splicing layer to obtain a first output image; wherein the three first convolutional layers include a first 3×3 convolutional layer, a first 9×1 tubular convolutional layer, and a first 16×1 tubular convolutional layer; Each feature reconstruction module includes three second convolutional layers, a second splicing layer and a third splicing layer, wherein the second splicing layer is used to splice the two second input images in the channel dimension direction to obtain a second intermediate image, the three second convolutional layers are used to process the second intermediate image to obtain three corresponding second feature maps, and the third splicing layer is used to splice the three second feature maps in the channel dimension direction to obtain a second output image; wherein the three second convolutional layers include a second 3×3 convolutional layer, a second 9×1 tubular convolutional layer and a second 16×1 tubular convolutional layer; The processing of each 9×1 tubular convolutional layer includes: The third input image is processed using a 9×1 convolution kernel to obtain an offset of each pixel in the third input image and record the offset as first pixel offset information; Using the first pixel offset information, fine-tuning the position of each pixel in the third input image to obtain a third intermediate image; Performing bilinear interpolation on the third intermediate image to obtain a third output image; The processing of each 16×1 tubular convolutional layer includes: Processing the fourth input image with a 16×1 convolution kernel to obtain an offset of each pixel in the fourth input image and record the offset as second pixel offset information; fine-adjusting the position of each pixel in the fourth input image by using the second pixel offset information to obtain a fourth intermediate image; Perform bilinear interpolation on the fourth intermediate image to obtain a fourth output image.

2. The method for automatically delineating the brain midline and detecting the offset according to claim 1, characterized in that: The step of calibrating the original brain CT image to obtain a calibrated brain CT image includes: Preprocessing the original brain CT image to obtain a brain CT image to be tested; Among them, the preprocessing methods include binarization and morphological closing operation; Extracting all boundary points on the maximum closed contour line from the brain CT image to be tested, and then performing ellipse fitting on all the boundary points to obtain an ellipse curve equation; Determining the rotation angle of the fitted ellipse according to the parameter information contained in the elliptic curve equation; Determining whether the rotation angle is zero; If yes, the original brain CT image is directly used as the calibration brain CT image; If not, determining the rotation matrix of the ellipse according to the parameter information and the rotation angle; The original brain CT image is multiplied by the rotation matrix to obtain a calibrated brain CT image.

3. The method for automatically delineating the brain midline and detecting the offset according to claim 1, characterized in that: Determining the offset of the predicted brain midline relative to the ideal brain midline comprises: In the feature image, all pixel points on the predicted brain center line are extracted to form a predicted point set, and all pixel points on the ideal brain center line are extracted to form an ideal point set; For each pixel in the prediction point set, the pixel is recorded as a prediction pixel and a circular neighborhood is created with the pixel as the center, and then all the pixels falling in the circular neighborhood are screened out from the ideal point set, and the number of all the pixels is used as the density value of the prediction pixel; A pixel point with the smallest density value is selected from the predicted point set and recorded as a target pixel point, and then the vertical distance between the target pixel point and the ideal brain midline is used as an offset.

4. The method for automatically delineating the brain midline and detecting the offset according to claim 1, characterized in that: The method further includes visualizing the target brain CT image and the offset.

5. A brain midline automatic delineation and offset detection system, characterized in that: The system comprises: The first module is used to obtain the original brain CT image; The second module is used to calibrate the original brain CT image to obtain a calibrated brain CT image; A third module is used to process the calibrated brain CT image using the trained brain midline extraction model to obtain a feature image containing only the predicted brain midline; A fourth module is used to superimpose the characteristic image and the calibrated brain CT image to obtain a target brain CT image that automatically outlines the predicted brain midline; A fifth module is used to automatically generate an ideal brain midline in the feature image according to two endpoints of the predicted brain midline, and then determine an offset of the predicted brain midline relative to the ideal brain midline; The brain midline extraction model includes an encoder and a decoder, the encoder includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module, a first downsampling layer, a second downsampling layer and a third downsampling layer, and the decoder includes a first feature reconstruction module, a second feature reconstruction module, a third feature reconstruction module, a first upsampling layer, a second upsampling layer and a third upsampling layer; The output end of the first feature extraction module is connected to the input end of the second feature extraction module and the input end of the third feature reconstruction module respectively through the first down-sampling layer, the output end of the second feature extraction module is connected to the input end of the third feature extraction module and the input end of the second feature reconstruction module respectively through the second down-sampling layer, the output end of the third feature extraction module is connected to the input end of the fourth feature extraction module and the input end of the first feature reconstruction module respectively through the third down-sampling layer, the output end of the fourth feature extraction module is connected to the input end of the first feature reconstruction module through the first up-sampling layer, the output end of the first feature reconstruction module is connected to the input end of the second feature reconstruction module through the second up-sampling layer, and the output end of the second feature reconstruction module is connected to the input end of the third feature reconstruction module through the third up-sampling layer; Each feature extraction module includes three first convolutional layers and a first splicing layer, wherein the first input image is processed by the three first convolutional layers to obtain three corresponding first feature maps, and the three first feature maps are spliced ​​in the channel dimension direction by the first splicing layer to obtain a first output image; wherein the three first convolutional layers include a first 3×3 convolutional layer, a first 9×1 tubular convolutional layer, and a first 16×1 tubular convolutional layer; Each feature reconstruction module includes three second convolutional layers, a second splicing layer and a third splicing layer, wherein the second splicing layer is used to splice the two second input images in the channel dimension direction to obtain a second intermediate image, the three second convolutional layers are used to process the second intermediate image to obtain three corresponding second feature maps, and the third splicing layer is used to splice the three second feature maps in the channel dimension direction to obtain a second output image; wherein the three second convolutional layers include a second 3×3 convolutional layer, a second 9×1 tubular convolutional layer and a second 16×1 tubular convolutional layer; The processing of each 9×1 tubular convolutional layer includes: The third input image is processed using a 9×1 convolution kernel to obtain an offset of each pixel in the third input image and record the offset as first pixel offset information; Using the first pixel offset information, fine-tuning the position of each pixel in the third input image to obtain a third intermediate image; Performing bilinear interpolation on the third intermediate image to obtain a third output image; The processing of each 16×1 tubular convolutional layer includes: Processing the fourth input image with a 16×1 convolution kernel to obtain an offset of each pixel in the fourth input image and record the offset as second pixel offset information; fine-adjusting the position of each pixel in the fourth input image by using the second pixel offset information to obtain a fourth intermediate image; Perform bilinear interpolation on the fourth intermediate image to obtain a fourth output image.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: The processor executes the computer program to implement the method for automatically delineating the brain midline and detecting the offset as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for automatically delineating the brain midline and detecting the offset as claimed in any one of claims 1 to 4 is implemented.

Images