Medical image analysis method, device and equipment of end side model and storage medium

Abstract

The application relates to a medical image analysis method, device and equipment of an end-side model and a storage medium. The method comprises the following steps: inputting an initial image into a downsampling module; performing downsampling on the input initial image by using the downsampling module to obtain a downsampling result output by the downsampling module; inputting the downsampling result into a deep convolution layer to obtain a feature image output by the deep convolution layer; performing global information simulation calculation on the feature image by using a low-dimensional state space model to obtain global context semantic information; inputting the global context semantic information into a transposed convolution layer to obtain a transposed convolution image output by the transposed convolution layer; inputting the downsampling result and the transposed convolution image into an upsampling module to perform upsampling to obtain an upsampling result output by the upsampling module; and obtaining a target image output by a trained end-side model based on the upsampling result. The application can improve the analysis efficiency of medical images.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method, apparatus, device and storage medium for medical image analysis of an end-side model. Background Technology

[0002] The rapid development of digital technology has greatly promoted the progress of digital healthcare and laid a solid information foundation for it. Digital healthcare based on artificial intelligence (AI) technology has gradually played a role in medical interventions. Currently, the main AI methods for medical image analysis include methods based on convolutional neural networks (CNNs) and methods based on self-attention mechanisms. These methods have promoted the intelligent development of medical image analysis and provided important assistance to doctors in clinical practice.

[0003] However, existing medical image analysis models have high computational complexity, resulting in slow processing speeds and difficulty in meeting the real-time requirements of medical practice. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for medical image analysis using an edge-side model, aiming to solve the problem that existing medical image analysis models have high computational complexity, resulting in slow processing speeds and difficulty in meeting the real-time requirements of medical applications.

[0005] In a first aspect, embodiments of this application provide a medical image analysis method using an edge-side model. The trained edge-side model includes a downsampling module, an upsampling module, and a feature extraction module. The feature extraction module includes a deep convolutional layer, a low-dimensional state-space model, and a transposed convolutional layer. The method includes:

[0006] Input the initial image into the downsampling module;

[0007] The input initial image is downsampled using the downsampling module to obtain the downsampling result output by the downsampling module;

[0008] The downsampling result is input into the deep convolutional layer to obtain the feature image output by the deep convolutional layer;

[0009] A low-dimensional state-space model is used to simulate and calculate global information in the feature image to obtain global contextual semantic information.

[0010] Global contextual semantic information is input into the transposed convolutional layer to obtain the transposed convolutional image output by the transposed convolutional layer;

[0011] The downsampling result and the transposed convolutional image are input into the upsampling module for upsampling, and the upsampling result output by the upsampling module is obtained.

[0012] Based on the upsampling results, the target image output by the trained edge model is obtained.

[0013] In some implementations, before using a low-dimensional state-space model to perform global information simulation calculations on the feature image to obtain global contextual semantic information, the method further includes:

[0014] The feature images are sorted in the forward direction to obtain the forward direction feature sequence;

[0015] The feature images are sorted in reverse order to obtain the reverse-direction feature sequence;

[0016] Sort the feature images according to the depth direction to obtain the depth direction feature sequence;

[0017] By using a low-dimensional state-space model to simulate and calculate global information from the feature image, global contextual semantic information is obtained, including:

[0018] Input the forward direction feature sequence, the reverse direction feature sequence, and the depth direction feature sequence into the low-dimensional state space model;

[0019] A low-dimensional state-space model is used to model the global information of the feature sequences in the forward direction, the reverse direction, and the depth direction, respectively, to obtain the global contextual semantic information output by the low-dimensional state-space model. The global contextual semantic information includes forward global information, reverse global information, and depth global information.

[0020] In some implementations, the method further includes, before inputting global context semantic information into the transposed convolutional layer:

[0021] The forward global information, the reverse global information, and the deep global information are fused to obtain the fusion matrix;

[0022] The fusion matrix is ​​linearly transformed to obtain the query vector Q, the key vector K, and the value vector V.

[0023] Input the transpose of the key vector K, the query vector Q, the value vector V, and the preset scaling factor into the self-attention layer to obtain the attention-enhanced features output by the self-attention layer.

[0024] Global contextual semantic information is input into the transposed convolutional layer to obtain the transposed convolutional image output by the transposed convolutional layer, including:

[0025] The attention-enhancing features are input into the transposed convolutional layer, resulting in the transposed convolutional output of the transposed convolutional layer.

[0026] In some implementations, the downsampling module includes an i-layer downsampling network layer;

[0027] The input initial image is downsampled using a downsampling module, and the downsampling result output by the downsampling module is obtained, including:

[0028] The initial image is input into the first downsampling network layer to obtain the first downsampling result output by the first downsampling network layer;

[0029] The downsampling result of the first layer is used as the input of the next downsampling network layer for repeated operation until the downsampling result of each downsampling network layer is obtained.

[0030] In some implementations, the upsampling module includes an i-th upsampling network layer. After obtaining the downsampling enhancement results corresponding to each downsampling network layer, the method further includes:

[0031] The downsampling results output by each downsampling network layer are enhanced separately to obtain the downsampling enhancement results corresponding to each downsampling network layer;

[0032] The downsampling enhancement results corresponding to each downsampling network layer are input into the feature extraction module to obtain the transposed convolutional image corresponding to each downsampling enhancement result output by the feature extraction module.

[0033] The downsampling result and the transposed convolutional image are input into the upsampling module for upsampling, resulting in the upsampling result output by the upsampling module, including:

[0034] The downsampling result output by the i-th downsampling network layer is fused with the transposed convolutional image corresponding to the i-th downsampling network layer and input into the 1-th upsampling network layer to obtain the upsampling result output by the 1-th upsampling network layer.

[0035] The upsampled result output from the first upsampled network layer and the transposed convolutional image corresponding to the (i-1)th downsampled network layer are used as inputs to the next upsampled network layer. This process is repeated until the upsampled result output from the i-th upsampled network layer is obtained.

[0036] In some implementations, the method further includes:

[0037] When i∈[1,2], the stride of both the depthwise convolutional layer and the transposed convolutional layer is configured to 4, and the kernel size of both the depthwise convolutional layer and the transposed convolutional layer is configured to 3.

[0038] When i∈[3,4], the stride of the deep convolutional layer is configured to 2, and the kernel size of the deep convolutional layer is configured to 1;

[0039] Configure the stride of the transposed convolutional layer to 2, the kernel size to 4, and the padding size to 1.

[0040] In some implementations, the downsampling result output by the downsampling network layer is enhanced to obtain the downsampling enhancement result corresponding to the downsampling network layer, including:

[0041] The downsampling result output by the downsampling network layer is subjected to a first depthwise convolution to obtain the first depthwise convolution result;

[0042] The downsampling result output by the downsampling network layer is subjected to the first pointwise convolution to obtain the first pointwise convolution result;

[0043] Perform a second depthwise convolution on the result of the first depthwise convolution to obtain the result of the second depthwise convolution;

[0044] The first pointwise convolution result is fused with the second depthwise convolution result to obtain the convolution result;

[0045] Perform a second pointwise convolution on the convolution result to obtain the second pointwise convolution result;

[0046] The downsampling result output from the downsampling network layer is fused with the result of the second pointwise convolution and input into the fully connected layer for linear transformation to obtain the downsampling enhancement result output by the fully connected layer.

[0047] Secondly, embodiments of this application also provide a medical image analysis apparatus for an end-side model, which includes a unit for performing the above-described method.

[0048] Thirdly, embodiments of this application also provide a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0049] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-described method.

[0050] This application provides a method, apparatus, device, and storage medium for medical image analysis using an edge-side model. The method includes: inputting an initial image into a downsampling module; downsampling the input initial image using the downsampling module to obtain a downsampling result output by the downsampling module; inputting the downsampling result into a deep convolutional layer to obtain a feature image output by the deep convolutional layer; performing global information simulation calculations on the feature image using a low-dimensional state-space model to obtain global contextual semantic information; inputting the global contextual semantic information into a transposed convolutional layer to obtain a transposed convolutional image output by the transposed convolutional layer; inputting the downsampling result and the transposed convolutional image into an upsampling module to obtain an upsampling result output by the upsampling module; and obtaining a target image output by a trained edge-side model based on the upsampling result.

[0051] In this embodiment, the downsampling result is input into a deep convolutional layer to obtain a feature image. The deep convolutional layer maps the feature image to a low-dimensional space. The feature map in the low-dimensional space has fewer channels and a smaller spatial size, which can directly reduce the computational cost of the convolution operation. Furthermore, by using a low-dimensional state space model to perform dimensionality reduction on the feature image, unimportant features can be removed, the most critical information can be retained, and unnecessary computation can be reduced. Therefore, by using a low-dimensional state space model, global information simulation and computation can be performed more efficiently in the low-dimensional space, capturing global context information without the need for complex computations in the high-dimensional space, thereby improving processing efficiency. Attached Figure Description

[0052] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0055] Figure 1 This is a flowchart illustrating a medical image analysis method for an end-side model provided in an embodiment of this application.

[0056] Figure 2 This is a flowchart illustrating the U-shaped sampling and prediction network framework provided in the embodiments of this application.

[0057] Figure 3 This is a flowchart illustrating the feature extraction module provided in an embodiment of this application.

[0058] Figure 4 This is a flowchart illustrating the GMP Block module provided in an embodiment of this application.

[0059] Figure 5 This is a schematic diagram of an embodiment of the computer device provided in this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0062] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0063] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0064] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0065] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0066] To address the technical problem that the high computational complexity of existing medical image analysis models leads to slow processing speeds and makes it difficult to meet the real-time requirements of medical applications, this application provides a medical image analysis method based on an edge-side model, which can improve the efficiency of medical image analysis.

[0067] Figure 1 This is a flowchart illustrating a medical image analysis method using an edge-side model provided in an embodiment of this application. The trained edge-side model includes a downsampling module, an upsampling module, and a feature extraction module. The feature extraction module includes a deep convolutional layer, a low-dimensional state-space model, and a transposed convolutional layer. The method includes:

[0068] Step 110: Input the initial image into the downsampling module.

[0069] Step 120: Use the downsampling module to downsample the input initial image to obtain the downsampling result output by the downsampling module.

[0070] Step 130: Input the downsampling result into the deep convolutional layer to obtain the feature image output by the deep convolutional layer.

[0071] Step 140: Use a low-dimensional state-space model to perform global information simulation calculation on the feature image to obtain global context semantic information.

[0072] Step 150: Input the global context semantic information into the transposed convolutional layer to obtain the transposed convolutional image output by the transposed convolutional layer.

[0073] Step 160: Input the downsampling result and the transposed convolutional image into the upsampling module for upsampling, and obtain the upsampling result output by the upsampling module.

[0074] Step 170: Based on the upsampling results, obtain the target image output by the trained end-side model.

[0075] Because feature maps in low-dimensional space have fewer channels and smaller spatial dimensions, the computational cost of convolution operations can be directly reduced. Furthermore, using a low-dimensional state-space model to reduce the dimensionality of the feature image can remove unimportant features, retaining only the most crucial information and reducing unnecessary computation. Therefore, this embodiment inputs the downsampling result into a deep convolutional layer to obtain the feature image. The deep convolutional layer then maps the feature image to a low-dimensional space. The low-dimensional state-space model allows for more efficient global information simulation and computation in this space, capturing global contextual information without the need for complex calculations in high-dimensional space. This improves processing efficiency and thus increases throughput.

[0076] Furthermore, since this embodiment captures global contextual semantic information and then maps low-dimensional features back to the original spatial dimension through transposed convolutional layers, the accuracy of image segmentation can be guaranteed.

[0077] Referring to a second embodiment of a medical image analysis method for end-side models provided in this application, the second embodiment may include the following steps:

[0078] Step 210: Input the initial image into the downsampling module.

[0079] Step 220: Use the downsampling module to downsample the input initial image and obtain the downsampling result output by the downsampling module.

[0080] Step 230: Input the downsampling result into the deep convolutional layer to obtain the feature image output by the deep convolutional layer.

[0081] Step 230 can be implemented using the following formula 1:

[0082] X sq =Dw i (X) a , Formula 1.

[0083] Among them, X sq X is the feature image, and X is the downsampling result.

[0084] Step 240: Sort the feature images in the forward direction to obtain the forward direction feature sequence; sort the feature images in the reverse direction to obtain the reverse direction feature sequence; sort the feature images in the depth direction to obtain the depth direction feature sequence.

[0085] Specifically, for the feature image X sq Sort the features in the forward direction (e.g., from top left to bottom right of the image) to obtain the forward direction feature sequence X. sqF .

[0086] For feature image X sq Sort in reverse direction (e.g., from bottom right to top left of the image) to obtain the reverse direction feature sequence X. sqR .

[0087] For feature image X sq Sort the data along the depth direction (e.g., from the first frame to the last frame pixel by pixel) to obtain the depth direction feature sequence X. sqS .

[0088] Step 250: Input the forward direction feature sequence, the reverse direction feature sequence, and the depth direction feature sequence into the low-dimensional state space model.

[0089] Step 260: Use a low-dimensional state space model to model the global information of the forward direction feature sequence, the reverse direction feature sequence, and the depth direction feature sequence respectively, and obtain the global context semantic information output by the low-dimensional state space model.

[0090] The global context semantic information includes forward global information, reverse global information, and deep global information.

[0091] Step 270: Input the global context semantic information into the transposed convolutional layer to obtain the transposed convolutional image output by the transposed convolutional layer.

[0092] In some embodiments, before step 270, which involves inputting global contextual semantic information into the transposed convolutional layer to obtain the transposed convolutional image output by the transposed convolutional layer, the following steps may also be included:

[0093] Step 271: Fuse the forward global information, the reverse global information, and the deep global information to obtain the fusion matrix.

[0094] Step 271 can be implemented using the following formula 2:

[0095]

[0096] Wherein, M(X) sqF ) represents positive global information, M(X) sqR) For reverse global information, M(X) sqS This represents deep global information. Let M be the fusion matrix, and M be the low-dimensional state-space model.

[0097] Step 272: Perform a linear transformation on the fusion matrix to obtain the query vector Q, the key vector K, and the value vector V.

[0098] Step 272 can be achieved using the following formula 3:

[0099]

[0100] Step 273: Input the transpose of the key vector K, the query vector Q, the value vector V, and the preset scaling factor into the self-attention layer to obtain the attention-enhanced features output by the self-attention layer.

[0101] Step 273 can be achieved using the following formula 4:

[0102]

[0103] in, X is the preset scaling factor. a For attention enhancement features, T is the transpose.

[0104] Based on this, step 270 can be to input the attention enhancement features into the transposed convolutional layer to obtain the transposed convolution result output by the transposed convolutional layer.

[0105] Specifically, formula 5 can be used:

[0106] X′=Tw i (X a ) a , Formula 5.

[0107] Where X′ is the transposed convolution result, i.e., the output of the feature extraction module Slim UNETR Block v2, and Tw i (·) a This is a transposed convolution.

[0108] Step 280: Input the downsampling result and the transposed convolutional image into the upsampling module for upsampling to obtain the upsampling result output by the upsampling module.

[0109] Step 290: Based on the upsampling results, obtain the target image output by the trained end-side model.

[0110] In some embodiments, the downsampling module may include an i-th downsampling network layer (where i can be 4), and steps 120 and 220 may specifically include the following steps:

[0111] Step 310: Input the initial image into the first downsampling network layer to obtain the first downsampling result output by the first downsampling network layer.

[0112] Step 320: Repeat the operation by using the downsampling result of the first layer as the input of the next downsampling network layer until the downsampling results of each downsampling network layer are obtained.

[0113] For example, the downsampling result of the first layer is input into the second layer of the downsampling network layer to obtain the second layer downsampling result output by the second layer of the downsampling network layer;

[0114] The result of the second layer downsampling is input into the third layer downsampling network layer to obtain the third layer downsampling result output by the third layer downsampling network layer;

[0115] The downsampling result of the third layer is input into the fourth layer downsampling network layer to obtain the fourth layer downsampling result output by the fourth layer downsampling network layer.

[0116] In this way, the downsampling result of the first layer is used as the input of the next downsampling network layer for repeated operation until the downsampling result output by each downsampling network layer is obtained.

[0117] In some implementations, the upsampling module includes an i-th upsampling network layer. After obtaining the downsampling enhancement results corresponding to each downsampling network layer in step 320, the method further includes:

[0118] Step 330: Enhance the downsampling results output by each downsampling network layer to obtain the downsampling enhancement results corresponding to each downsampling network layer.

[0119] For example, the downsampling results of the first layer, the second layer, the third layer, and the fourth layer can be enhanced to obtain the enhanced results of the first layer, the second layer, the third layer, and the fourth layer.

[0120] In some implementations, step 330 may include the following steps:

[0121] Step 331: Perform a first depthwise convolution on the downsampling result output by the downsampling network layer to obtain the first depthwise convolution result.

[0122] Step 332: Perform the first pointwise convolution on the downsampling result output by the downsampling network layer to obtain the first pointwise convolution result.

[0123] Step 333: Perform a second depthwise convolution on the result of the first depthwise convolution to obtain the result of the second depthwise convolution.

[0124] Step 334: Fuse the first pointwise convolution result with the second depthwise convolution result to obtain the convolution result.

[0125] Step 335: Perform a second pointwise convolution on the convolution result to obtain the second pointwise convolution result.

[0126] Step 336: Fuse the downsampling result output from the downsampling network layer with the second pointwise convolution result and input it into the fully connected layer for linear transformation to obtain the downsampling enhancement result output from the fully connected layer.

[0127] For steps 331-336, the following formula 6 can be used:

[0128] X = FC i (X g +·PW i (PW i (X g )+·DW i (DW i (X g )))), Formula 6.

[0129] Among them, X g DW represents the downsampling result output by the downsampling network layer. i(X g ) represents the result of the first depthwise convolution, PW i (X g ) represents the first pointwise convolution result, DW i (DW i (X g The result of the second depthwise convolution is PW. i (X g )+·DW i (DW i (X g )) represents the convolution result, PW i (PW i (X g )+·DW i (DW i (X g ))) represents the result of the second pointwise convolution, X represents the downsampling enhancement result of the fully connected layer, i.e., the output result of the GMP Block, and i represents the feature map of the i-th downsampling layer being processed. FC i This represents a fully connected layer.

[0130] Among them, PW i (.) and DW i (.) represents pointwise convolution with a kernel size of 1 and depthwise convolution with a kernel size of 3, respectively.

[0131] In some implementations, instance normalization and adaptive activation function processing can be performed after pointwise convolution and depthwise convolution operations. Specifically, when i∈[1,2], the adaptive activation function can be set to the GELU function, and when i∈[3,4], the adaptive activation function can be set to the Swish function.

[0132] In addition, fully connected layers also apply adaptive activation functions. Similarly, when i∈[1,2], the adaptive activation function can be set to the GELU function, and when i∈[3,4], the adaptive activation function can be set to the Swish function.

[0133] Step 340: Input the downsampling enhancement results corresponding to each downsampling network layer into the feature extraction module to obtain the transposed convolutional image corresponding to each downsampling enhancement result output by the feature extraction module.

[0134] When i∈[1,2], the stride of both the depthwise convolutional layer and the transposed convolutional layer in the feature extraction module is configured to 4, and the kernel size of both the depthwise convolutional layer and the transposed convolutional layer is configured to 3.

[0135] When i∈[3,4], the stride of the deep convolutional layer is configured to 2, and the kernel size of the deep convolutional layer is configured to 1.

[0136] Configure the stride of the transposed convolutional layer to 2, the kernel size to 4, and the padding size to 1.

[0137] Step 350: The downsampling result output by the i-th downsampling network layer is fused with the transposed convolutional image corresponding to the i-th downsampling network layer and input into the 1-th upsampling network layer to obtain the upsampling result output by the 1-th upsampling network layer.

[0138] Step 360: Repeat the operation by using the upsampling result output by the first upsampling network layer and the transposed convolutional image corresponding to the (i-1)th downsampling network layer as the input to the next upsampling network layer until the upsampling result output by the i-th upsampling network layer is obtained.

[0139] For steps 350 and 360, please refer to... Figure 2 Specifically, it can include the following processes.

[0140] 1) Normalize the downsampling results of the 4th layer to obtain the normalized result;

[0141] 2) Perform a deconvolution operation on the normalized result to obtain the deconvolution result;

[0142] 3) Input the deconvolution result and the downsampling enhancement result of the 4th layer into the upsampling network layer of the 1st layer to obtain the upsampling result of the 1st layer output by the upsampling network layer;

[0143] 4) Input the upsampling result of the first layer and the downsampling enhancement result of the third layer into the second layer upsampling network layer to obtain the second layer upsampling result output by the second layer upsampling network layer;

[0144] 5) Input the upsampling result of the second layer and the downsampling enhancement result of the second layer into the third layer upsampling network layer to obtain the third layer upsampling result output by the third layer upsampling network layer;

[0145] 6) Input the upsampling result of the third layer and the downsampling enhancement result of the first layer into the fourth layer upsampling network layer to obtain the fourth layer upsampling result output by the fourth layer upsampling network layer.

[0146] In this way, the upsampled result output by the first upsampled network layer and the transposed convolutional image corresponding to the (i-1)th downsampled network layer are used as the input of the next upsampled network layer for repeated operation until the upsampled result output by the i-th upsampled network layer is obtained.

[0147] After obtaining the upsampling result of the 4th layer, it can be input into a separate convolutional layer for depth convolution to obtain a depth convolution image. Then, the Sigmoid activation function is used to calculate and predict the depth convolution image to obtain the analysis result, i.e., the prediction result f(X).input );

[0148] Then, by calculating the prediction result f(X) input The similarity and error values ​​between the prediction and the true value Y are used to evaluate the correctness of the prediction results.

[0149] Based on the above embodiments, combined with Figures 2-4 The medical image analysis model 100 provided in this application, namely SlimUNETRv2, mainly includes a U-shaped sampling and prediction network framework, a feature extraction module SlimUNETR Block v2, and a GMP Block module.

[0150] The U-shaped sampling and prediction network framework includes a downsampling module, an upsampling module, and a separate convolutional layer segment head, such as... Figure 2 As shown, the U-shaped sampling and prediction network framework can specifically include a four-stage downsampling process and a corresponding four-stage upsampling process.

[0151] Specifically, let the initial input image be a three-dimensional medical image X. input (W*H*D*C), where W, H, D, and C represent the width, height, depth, and number of channels of the initial image, respectively. The i-th downsampling stage of this U-shaped sampling and prediction network framework downsamples the input initial image to X through depthwise convolution. down The dimensions are W / 2 i+1 *H / 2 i+1 *D / 2 i+1 *C i , where C i ∈[64,128,256,512]. Subsequently, the corresponding upsampling stage upsamples the image to X using deconvolution. up Its size is related to the X of the corresponding stage. down Similarly, the final analysis result f(X) is calculated using a single convolutional layer, the segment head, and the sigmoid activation function. input Meanwhile, the U-shaped sampling and prediction network framework uses skip connections to link corresponding sampling results during downsampling and upsampling processes to enhance the final analysis performance.

[0152] The feature extraction module SlimUNETR Block v2 can include deep convolutional layers, a low-dimensional state-space model, and transposed convolutional layers. The low-dimensional state-space model employs a sparse state-space computation mechanism, providing low-computational feature extraction capabilities for high-performance medical image analysis. Its specific computational framework is as follows: Figure 3As shown. Specifically, this feature extraction module is deployed within the connections between the downsampling stages and the corresponding upsampling stages of the U-shaped sampling and prediction network framework.

[0153] The GMP Block module, based on differentiated stride convolution, is used for feature smoothing and spatial information supplementation of sampling results. Its specific computational framework is as follows: Figure 4 As shown. Specifically, the GMP Block module applies a set of convolutional layers and fully connected layers with different strides to enhance the capture of local information.

[0154] Assume the downsampling result received by the GMP Block module is X. g The output result is X, and its calculation can be found in Formula 6 above.

[0155] In some embodiments, the medical image analysis model SlimUNETR v2, during training, can use the predicted results f(X) obtained from training. input The index is calculated by comparing the true value Y of the image analysis results (Y is the result of the lesion / lesion analysis accurately judged by the doctor).

[0156] Specifically, the correctness of the medical image analysis model's analysis task can be evaluated using the following two main indicators:

[0157] 1) Dice coefficient: This value represents the result of the method analysis f(X) input The similarity between the actual value Y and the true value Y can be found in Formula 7 below:

[0158]

[0159] 2) HD95 error: This value represents the method analysis result f(X) input The error boundary between the true value Y and the actual value Y can be found in Formula 8 below:

[0160] HD·95=·percentile(distances(f(X input ),Y),95%), Formula 8.

[0161] Where distances(·,·) represents the set of distances between the closest points in two sets of analysis points, and percentile(,95%) represents the 95th percentile of the above distance set.

[0162] In summary, the feature extraction module SlimUNETR v2 maps the input features (i.e., the downsampling results output by each downsampling network layer) to a low-dimensional space (i.e., a low-dimensional state space model) through deep convolutional layers. It then performs three-way global information exchange in this low-dimensional space to capture global contextual semantic information. Finally, a transposed convolutional layer maps the low-dimensional features back to the original spatial dimension. This global information exchange within the low-dimensional state space model ensures segmentation accuracy while improving the efficiency of medical image analysis and processing.

[0163] Corresponding to the above-described end-to-end model medical image analysis method, this application also provides an end-to-end model medical image analysis device. This end-to-end model medical image analysis device includes a unit for executing the above-described end-to-end model medical image analysis method, and can be configured in a desktop computer, tablet computer, laptop computer, or other terminal.

[0164] like Figure 5 As shown in the figure, this application provides a computer device including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0165] Memory 113 is used to store computer programs;

[0166] In one embodiment of this application, the processor 111, when executing a program stored in the memory 113, implements the medical image analysis method of the end-side model provided in any of the foregoing method embodiments, including:

[0167] Input the initial image into the downsampling module;

[0168] The input initial image is downsampled using the downsampling module to obtain the downsampling result output by the downsampling module;

[0169] The downsampling result is input into the deep convolutional layer to obtain the feature image output by the deep convolutional layer;

[0170] A low-dimensional state-space model is used to simulate and calculate global information in the feature image to obtain global contextual semantic information.

[0171] Global contextual semantic information is input into the transposed convolutional layer to obtain the transposed convolutional image output by the transposed convolutional layer;

[0172] The downsampling result and the transposed convolutional image are input into the upsampling module for upsampling, and the upsampling result output by the upsampling module is obtained.

[0173] Based on the upsampling results, the target image output by the trained edge model is obtained.

[0174] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0175] Therefore, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the medical image analysis method for the end-side model as provided in any of the foregoing method embodiments. This includes:

[0176] Input the initial image into the downsampling module;

[0177] The input initial image is downsampled using the downsampling module to obtain the downsampling result output by the downsampling module;

[0178] The downsampling result is input into the deep convolutional layer to obtain the feature image output by the deep convolutional layer;

[0179] A low-dimensional state-space model is used to simulate and calculate global information in the feature image to obtain global contextual semantic information.

[0180] Global contextual semantic information is input into the transposed convolutional layer to obtain the transposed convolutional image output by the transposed convolutional layer;

[0181] The downsampling result and the transposed convolutional image are input into the upsampling module for upsampling, and the upsampling result output by the upsampling module is obtained.

[0182] Based on the upsampling results, the target image output by the trained edge model is obtained.

[0183] Storage media are physical, non-transitory storage media, such as USB flash drives, external hard drives, read-only memory (ROM), magnetic disks, or optical disks—various physical storage media capable of storing program code. Computer-readable storage media can be non-volatile or volatile.

[0184] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0185] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0186] The steps in the methods of this application embodiment can be adjusted, merged, or deleted according to actual needs. The units in the apparatus of this application embodiment can be merged, divided, or deleted according to actual needs. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0187] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application.

[0188] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0189] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Since these modifications and variations fall within the scope of the claims and their equivalents, this application also intends to include these modifications and variations.

[0190] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for medical image analysis using an end-side model, characterized in that, The trained edge model includes a downsampling module, an upsampling module, and a feature extraction module. The feature extraction module includes a deep convolutional layer, a low-dimensional state-space model, and a transposed convolutional layer. The method includes: The initial image is input into the downsampling module; The input initial image is downsampled using the downsampling module to obtain the downsampling result output by the downsampling module; The downsampling result is input into the deep convolutional layer to obtain the feature image output by the deep convolutional layer; The feature image is simulated and calculated using the low-dimensional state-space model to obtain global contextual semantic information. The global context semantic information is input into the transposed convolutional layer to obtain the transposed convolutional image output by the transposed convolutional layer; The downsampling result and the transposed convolutional image are input into the upsampling module for upsampling to obtain the upsampling result output by the upsampling module; Based on the upsampling results, the target image output by the trained edge model is obtained; Before using the low-dimensional state space model to perform global information simulation calculation on the feature image to obtain global context semantic information, the method further includes: The feature images are sorted in the forward direction to obtain a forward direction feature sequence; The feature images are sorted in reverse order to obtain a reverse-direction feature sequence; The feature images are sorted according to the depth direction to obtain a depth direction feature sequence; The step of using the low-dimensional state-space model to perform global information simulation calculations on the feature image to obtain global contextual semantic information includes: The forward direction feature sequence, the reverse direction feature sequence, and the depth direction feature sequence are input into the low-dimensional state space model; The low-dimensional state space model is used to model the global information of the forward direction feature sequence, the reverse direction feature sequence and the depth direction feature sequence respectively, to obtain the global context semantic information output by the low-dimensional state space model. The global context semantic information includes forward global information, reverse global information and depth global information. The downsampling module includes an i-layer downsampling network layer; The step of downsampling the input initial image using the downsampling module to obtain the downsampling result output by the downsampling module includes: The initial image is input into the first downsampling network layer to obtain the first downsampling result output by the first downsampling network layer; The downsampling result of the first layer is used as the input of the next downsampling network layer and the operation is repeated until the downsampling result output by each downsampling network layer is obtained; The upsampling module includes an i-layer upsampling network layer. After obtaining the downsampling enhancement results corresponding to each downsampling network layer, the method further includes: The downsampling results output by each downsampling network layer are enhanced separately to obtain the downsampling enhancement results corresponding to each downsampling network layer; The downsampling enhancement results corresponding to each downsampling network layer are input into the feature extraction module to obtain the transposed convolutional image corresponding to each downsampling enhancement result output by the feature extraction module. The step of inputting the downsampling result and the transposed convolutional image into the upsampling module for upsampling to obtain the upsampling result output by the upsampling module includes: The downsampling result output by the i-th downsampling network layer is fused with the transposed convolutional image corresponding to the i-th downsampling network layer and input into the 1-th upsampling network layer to obtain the upsampling result output by the 1-th upsampling network layer. The upsampling result output by the first upsampling network layer and the transposed convolutional image corresponding to the (i-1)th downsampling network layer are used as the input of the next upsampling network layer and the operation is repeated until the upsampling result output by the i-th upsampling network layer is obtained. The method further includes: When i∈[1,2], the stride of both the depthwise convolutional layer and the transposed convolutional layer is configured to 4, and the kernel size of both the depthwise convolutional layer and the transposed convolutional layer is configured to 3. When i∈[3,4], the stride of the deep convolutional layer is configured to 2, and the kernel size of the deep convolutional layer is configured to 1; The stride of the transposed convolutional layer is configured to 2, the kernel size to 4, and the padding size to 1.

2. The method according to claim 1, characterized in that, Before inputting the global context semantic information into the transposed convolutional layer, the method further includes: The forward global information, the reverse global information, and the deep global information are fused to obtain a fusion matrix; The fusion matrix is ​​linearly transformed to obtain the query vector Q, the key vector K, and the value vector V. The transpose of the key vector K, the query vector Q, the value vector V, and the preset scaling factor are input into the self-attention layer to obtain the attention-enhanced features output by the self-attention layer. The step of inputting the global context semantic information into the transposed convolutional layer to obtain the transposed convolutional image output by the transposed convolutional layer includes: The attention enhancement feature is input into the transposed convolutional layer to obtain the transposed convolution result output by the transposed convolutional layer.

3. The method according to claim 1, characterized in that, The downsampling result output from the downsampling network layer is enhanced to obtain the corresponding downsampling enhancement result of the downsampling network layer, including: The downsampling result output by the downsampling network layer is subjected to a first depthwise convolution to obtain the first depthwise convolution result; The downsampling result output by the downsampling network layer is subjected to the first pointwise convolution to obtain the first pointwise convolution result; Perform a second depthwise convolution on the first depthwise convolution result to obtain the second depthwise convolution result; The first pointwise convolution result is fused with the second depthwise convolution result to obtain the convolution result; The convolution result is then subjected to a second pointwise convolution to obtain the second pointwise convolution result; The downsampling result output by the downsampling network layer is fused with the second pointwise convolution result and input into the fully connected layer for linear transformation to obtain the downsampling enhancement result output by the fully connected layer.

4. A medical image analysis device for an end-side model, characterized in that, Includes a unit for performing the method as described in any one of claims 1-3.

5. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in any one of claims 1-3.

Images