Deep learning-based knee joint image segmentation methods, systems, and electronic devices

Abstract

This invention provides a deep learning-based method, system, and electronic device for knee joint image segmentation. The method includes: inputting a knee joint image to be segmented into a knee joint image segmentation model to obtain a segmented knee joint image output by the knee joint image segmentation model; the knee joint image segmentation model includes a multi-layer network and a spatial channel dual attention module, wherein the multi-layer network includes at least one target layer, which is constructed based on an encoder-decoder network and an attention-gated neural network. This invention can effectively learn various detailed features in the knee joint image to be segmented using the knee joint image segmentation model, obtaining a high-precision segmented knee joint image. It can effectively complete image segmentation tasks, improve the accuracy of segmentation results, and achieve high-precision image segmentation results even in cases where the knee joint has rich detailed features.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a knee joint image segmentation method, system, and electronic device based on deep learning. Background Technology

[0002] With the rapid development of knee joint surgery and AI technology, deep learning-based knee joint segmentation algorithms are beginning to emerge in the medical field. Deep learning is a method of pixel classification, and the most representative existing technology is the U-Net network model, which is one of the commonly used deep learning algorithms in medical image segmentation.

[0003] However, due to the diversity of knee joint diseases, such as fractures and arthritis, there are segmentation targets of different sizes in knee joint images. The traditional U-Net network model cannot perform image segmentation tasks well, and the accuracy of the segmentation results is not high, especially when the knee joint has rich detailed features, the segmentation deviation is large. Summary of the Invention

[0004] This invention provides a deep learning-based knee joint image segmentation method, system, and electronic device to address the shortcomings of the traditional U-Net network model in the prior art, which cannot perform image segmentation tasks well and has low accuracy, especially in cases where the knee joint has rich detailed features and large segmentation deviations.

[0005] This invention provides a deep learning-based method for knee joint image segmentation, comprising:

[0006] The knee joint image to be segmented is input into the knee joint image segmentation model to obtain the knee joint segmentation image output by the knee joint image segmentation model;

[0007] The knee joint image segmentation model includes a multi-layer network and a spatial channel dual attention module. The multi-layer network includes at least one target layer, which is constructed based on an encoder-decoder network and an attention gate neural network.

[0008] The attention-gated neural network is used to fuse features between the encoded image of the target layer and the decoded image of the next layer below the target layer to generate a first feature fusion image; the encoded image of the target layer is determined based on the knee joint image to be segmented.

[0009] The spatial channel dual attention module is used to fuse channel attention features and spatial attention features in the target feature stitched image to obtain the knee joint segmentation image; the target feature stitched image is determined based on the second feature fusion image output by each target layer; the second feature fusion image of the target layer is determined based on the first feature fusion image of the target layer and the decoded image of the next layer below the target layer.

[0010] According to the present invention, a deep learning-based knee joint image segmentation method is provided, wherein the multi-layer network sequentially includes a first layer, the at least one target layer, and a last layer, wherein the first layer and the last layer are both encoding / decoding networks;

[0011] The knee joint image segmentation model is specifically used for:

[0012] The knee joint image to be segmented is encoded sequentially through the encoding networks of each layer in the multi-layer network to obtain the encoded images of each layer in the multi-layer network;

[0013] The encoded image of the last layer is subjected to three-dimensional convolution processing to obtain the decoded image of the last layer;

[0014] The second feature fusion image of the target layer is determined using the attention-gated neural network based on the encoded image of the target layer and the decoded image of the next layer after the target layer.

[0015] The decoded image of the first layer is determined by the decoding network in the first layer, based on the encoded image of the first layer and the decoded image of the next target layer.

[0016] A feature stitching operation is performed on the second feature fusion image of each target layer, the decoded image of the first layer, and the decoded image of the last layer to obtain the target feature stitched image;

[0017] The spatial channel dual attention module is used to fuse channel attention features and spatial attention features in the target feature stitched image to obtain the knee joint segmentation image.

[0018] According to a deep learning-based knee joint image segmentation method provided by the present invention, the spatial channel dual attention module is specifically used for:

[0019] Based on the channel attention mechanism, channel attention features are extracted from the target feature stitched image to obtain the target channel attention feature image;

[0020] Based on the spatial attention mechanism, spatial attention features are extracted from the target feature stitched image to obtain the target spatial attention feature image;

[0021] The target channel attention feature image and the target spatial attention feature image are superimposed to obtain a target feature fusion image. The target feature fusion image is then subjected to one-dimensional convolution to obtain the knee joint segmentation image.

[0022] According to a deep learning-based knee joint image segmentation method provided by the present invention, the method involves extracting channel attention features from the target feature stitched image based on a channel attention mechanism to obtain a target channel attention feature image, comprising:

[0023] The target feature stitched image is subjected to one-dimensional convolution processing to obtain a first feature image, and the target feature stitched image is subjected to three-dimensional convolution processing to obtain a second feature image;

[0024] The first feature image and the second feature image are superimposed to obtain the third feature image;

[0025] Perform feature classification on the third feature image to obtain a first attention coefficient array;

[0026] The first attention coefficient array is multiplied by the first feature image to obtain the target channel attention feature image.

[0027] According to a deep learning-based knee joint image segmentation method provided by the present invention, the step of extracting spatial attention features from the target feature stitched image based on a spatial attention mechanism to obtain a target spatial attention feature image includes:

[0028] The target feature stitched image is subjected to one-dimensional convolution processing to obtain the first feature image;

[0029] Perform feature classification on the first feature image to obtain the fourth feature image;

[0030] Perform a dot product operation on the first feature image and the fourth feature image to obtain the fifth feature image;

[0031] The fifth feature image is encoded to obtain the sixth feature image;

[0032] The first feature image and the sixth feature image are superimposed to obtain the target space attention feature image.

[0033] According to a deep learning-based knee joint image segmentation method provided by the present invention, the step of determining a second feature fusion image of the target layer based on the encoded image of the target layer and the decoded image of the next layer after the target layer through the attention-gated neural network includes:

[0034] The encoded image of the target layer and the decoded image of the next layer of the target layer are input into the attention gate neural network to obtain the first feature fusion image of the target layer output by the attention gate neural network;

[0035] A feature concatenation operation is performed on the first feature fusion image of the target layer and the decoded image of the next layer of the target layer to obtain the second feature fusion image of the target layer.

[0036] According to the deep learning-based knee joint image segmentation method provided by the present invention, the attention-gating neural network is specifically used for:

[0037] The encoded image of the target layer and the decoded image of the next layer below the target layer are superimposed to obtain the seventh feature image;

[0038] Perform feature classification on the seventh feature image to obtain a second attention coefficient array;

[0039] The second attention coefficient array is multiplied by the encoded image of the target layer to obtain the eighth feature image.

[0040] The eighth feature image is subjected to three-dimensional convolution processing to obtain the first feature fusion image of the target layer.

[0041] This invention also provides a deep learning-based knee joint image segmentation system, comprising:

[0042] The image segmentation module is used to input the knee joint image to be segmented into the knee joint image segmentation model to obtain the knee joint segmented image output by the knee joint image segmentation model;

[0043] The knee joint image segmentation model includes a multi-layer network and a spatial channel dual attention module. The multi-layer network includes at least one target layer, which is constructed based on an encoder-decoder network and an attention gate neural network.

[0044] The attention-gated neural network is used to fuse features between the encoded image of the target layer and the decoded image of the next layer below the target layer to generate a first feature fusion image; the encoded image of the target layer is determined based on the knee joint image to be segmented.

[0045] The spatial channel dual attention module is used to fuse channel attention features and spatial attention features in the target feature stitched image to obtain the knee joint segmentation image; the target feature stitched image is determined based on the second feature fusion image output by each target layer; the second feature fusion image of the target layer is determined based on the first feature fusion image of the target layer and the decoded image of the next layer below the target layer.

[0046] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the deep learning-based knee joint image segmentation method described above.

[0047] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the deep learning-based knee joint image segmentation method as described above.

[0048] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the deep learning-based knee joint image segmentation method as described above.

[0049] The present invention provides a deep learning-based knee joint image segmentation method, system, and electronic device. This method utilizes an attention-gated neural network to fuse the encoding and decoding information of the knee joint image to be segmented. Then, a spatial channel dual-attention module is used to fuse multi-scale information across both spatial and channel dimensions for all features output from each network layer. A knee joint image segmentation model is then constructed using the encoding / decoding network, the attention-gated neural network, and the spatial channel dual-attention module. Inputting the knee joint image to be segmented into this model allows it to effectively learn various detailed features within the image, resulting in a high-precision segmented knee joint image. This approach effectively completes image segmentation tasks, improving the accuracy of the segmentation results. Even in scenarios with rich knee joint details, high-precision image segmentation results can still be achieved. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating the deep learning-based knee joint image segmentation method provided by the present invention.

[0052] Figure 2 This is a schematic diagram of the AG neural network structure in the deep learning-based knee joint image segmentation method provided by the present invention;

[0053] Figure 3 This is a schematic diagram of the network structure of the SCA module in the deep learning-based knee joint image segmentation method provided by the present invention;

[0054] Figure 4 This is a schematic diagram of the network model structure in the deep learning-based knee joint image segmentation method provided by the present invention;

[0055] Figure 5 This is a schematic diagram of the structure of the deep learning-based knee joint image segmentation system provided by the present invention;

[0056] Figure 6 This is a schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0058] The following combination Figures 1-6 This invention describes a deep learning-based knee joint image segmentation method, system, and electronic device.

[0059] Figure 1 This is a flowchart illustrating the deep learning-based knee joint image segmentation method provided by the present invention, as shown below. Figure 1 As shown, it includes: step 110.

[0060] Step 110: Input the knee joint image to be segmented into the knee joint image segmentation model to obtain the knee joint segmentation image output by the knee joint image segmentation model;

[0061] The knee joint image segmentation model includes a multi-layer network and a spatial channel-dual attention (SCA) module. The multi-layer network includes at least one target layer, which is constructed based on an encoder-decoder network and an attention gate (AG) neural network.

[0062] An AG neural network is used to fuse features between the encoded image of the target layer and the decoded image of the next layer below the target layer to generate a first feature fusion image; the encoded image of the target layer is determined based on the knee joint image to be segmented.

[0063] The SCA module is used to fuse channel attention features and spatial attention features into the target feature stitched image to obtain the knee joint segmentation image; the target feature stitched image is determined based on the second feature fusion image output by each target layer; the second feature fusion image of the target layer is determined based on the first feature fusion image of the target layer and the decoded image of the next layer below the target layer.

[0064] Specifically, the knee joint image to be segmented described in the embodiments of the present invention refers to a medical image obtained by radiological diagnosis of a patient's damaged knee joint, such as a knee CT image.

[0065] Understandably, in accordance with the requirements of preoperative planning and personalized prosthesis design for knee surgery, the parts to be segmented in the CT image of the knee joint can include the femur, femoral cartilage, tibia, and tibial cartilage, totaling four categories.

[0066] The knee joint segmentation image described in this embodiment of the invention refers to the segmentation result image corresponding to the knee joint image to be segmented, obtained by extracting and fusing features from the knee joint image to be segmented using a knee joint image segmentation model. Specifically, it can be represented as a binary black and white image, where black is the image background and white represents different types of knee joint bone images obtained through segmentation.

[0067] The first feature fusion image described in this embodiment of the invention refers to an image obtained by fusing the encoded and decoded information features of the encoded image of the target layer and the decoded image of the next layer after upsampling.

[0068] The second feature fusion image described in this embodiment of the invention refers to the decoded image obtained by performing a feature concat operation on the first feature fusion image output by the target layer and the decoded image of the next layer after upsampling.

[0069] The target feature stitched image described in the embodiments of the present invention can be obtained by performing Concat processing on the second feature fusion image output by each target layer.

[0070] In an embodiment of the present invention, the knee joint image segmentation model may include a multi-layer network and an SCA module. The multi-layer network includes at least one target layer, and each target layer is constructed based on an encoder-decoder network and an AG neural network.

[0071] The AG neural network is used to fuse features between the encoded image of each target layer and the upsampled decoded image of the next layer below the target layer to generate the first feature fusion image. The SCA module performs multi-dimensional feature fusion of channel attention features and spatial attention features on the target feature stitched image to obtain the knee joint segmentation image.

[0072] Specifically, in the embodiments of the present invention, after the knee joint image to be segmented is input into the knee joint image segmentation model, the knee joint image to be segmented can be first encoded at multiple scales by the encoding and decoding network pre-arranged in each target layer of the knee joint image segmentation model to obtain the encoded images output by each target layer.

[0073] Then, a first feature fusion image is generated by fusing the encoded image of the target layer and the upsampled decoded image of the next layer below the target layer using an AG neural network. Next, multi-scale feature fusion is performed on the first feature fusion image and the upsampled decoded image of the next layer below the target layer using a Concat operation in the decoding network. The multi-scale feature fusion image is then Concatted again to obtain the decoded images of each target layer, resulting in the second feature fusion image of each target layer. Finally, a Concat operation is performed again on the second feature fusion images of each target layer to obtain the target feature stitched image.

[0074] Finally, the SCA module is used to fuse multi-scale information in both spatial and channel dimensions of the target feature stitched image containing all features. Finally, dimensional operations are performed on the fusion result to obtain the knee joint segmentation image.

[0075] The deep learning-based knee joint image segmentation method of this invention fuses the encoding and decoding information of the knee joint image to be segmented using an attention-gated neural network, and then uses a spatial channel dual attention module to fuse multi-scale information between spatial and channel dimensions of all features output by each layer of the network. A knee joint image segmentation model is then constructed using the encoding / decoding network, the attention-gated neural network, and the spatial channel dual attention module. By inputting the knee joint image to be segmented into the knee joint image segmentation model, the model can effectively learn various detailed features in the knee joint image to be segmented, obtaining a high-precision segmented knee joint image. This method can effectively complete the image segmentation task, improve the accuracy of the segmentation results, and achieve high-precision image segmentation results even in cases where the knee joint has rich detailed features.

[0076] Based on the above embodiments, the multilayer network sequentially includes a first layer, at least one target layer, and a last layer, wherein both the first layer and the last layer are encoding / decoding networks.

[0077] The knee joint image segmentation model is specifically used for:

[0078] The knee joint image to be segmented is encoded sequentially through the encoding networks of each layer in the multi-layer network to obtain the encoded images of each layer of the multi-layer network;

[0079] The encoded image of the last layer is subjected to three-dimensional convolution processing to obtain the decoded image of the last layer;

[0080] Using an AG neural network, the second feature fusion image of the target layer is determined based on the encoded image of the target layer and the decoded image of the next layer below the target layer;

[0081] The decoded image of the first layer is determined by the decoding network in the first layer, based on the encoded image of the first layer and the decoded image of the next target layer.

[0082] The second feature fusion image of each target layer, the decoded image of the first layer, and the decoded image of the last layer are subjected to feature stitching operation to obtain the target feature stitched image;

[0083] The SCA module is used to fuse channel attention features and spatial attention features in the target feature stitched image to obtain a knee joint segmentation image.

[0084] Specifically, the multilayer network described in this embodiment of the invention may sequentially include a first layer, at least one target layer, and a last layer, wherein both the first layer and the last layer may be configured as an encoding / decoding network. That is, one or more target layers can be deployed according to actual needs, and each target layer is constructed based on an encoding / decoding network and an AG neural network.

[0085] In embodiments of the present invention, the more target layers there are, the richer the features of the extracted and fused knee joint images become, which is more conducive to the network learning the detailed features of the diseased knee joint and the higher the accuracy and precision of the image segmentation results. However, the computational load will also gradually increase, affecting the segmentation efficiency. Therefore, a suitable number of target layers can be set according to actual computational needs to balance image segmentation accuracy and image segmentation efficiency and achieve the optimal image segmentation effect.

[0086] In embodiments of the present invention, the encoding operation may specifically include two types of encoding operations. The first type of operation may be a three-dimensional convolution processing operation, which includes sequentially performing a 3D convolution (Conv3×3) operation, a batch normalization (BN) operation, and a ReLU activation function operation. The second type of operation may be a max pooling (MP) operation. Similarly, the decoding operation may also specifically include two types of operations: one is a three-dimensional convolution processing operation, and the other is a feature concatenation operation, such as a concat operation.

[0087] In an embodiment of the present invention, after the knee joint image to be segmented is input into the knee joint image segmentation model, the knee joint image to be segmented is encoded through the encoding network of each layer in the multi-layer network. The above two types of encoding operations are performed on the knee joint image to be segmented to obtain the encoded images of each layer in the multi-layer network.

[0088] More specifically, the knee joint image to be segmented passes through the encoding network in the first layer. The image undergoes a 3D convolution (Conv3×3), batch normalization (BN), and ReLU activation sequentially to obtain the encoded image of the first layer. By performing an MP operation on the encoded image of the first layer, feature compression is achieved. This compressed image is then input into the encoding network of the next target layer. This encoding process is repeated, sequentially passing through the encoding networks of each target layer up to the last layer, to obtain the encoded images of each target layer and the last layer.

[0089] Furthermore, the encoded image of the last layer is subjected to a three-dimensional convolution operation through the decoding network in the last layer to obtain the decoded image of the last layer.

[0090] Furthermore, using an AG neural network, the encoded image of the target layer and the upsampled decoded image of the next layer below the target layer are fused using the encoded and decoded information to obtain the first feature fusion image of the target layer. By performing a concat operation on this first feature fusion image and the upsampled decoded image of the next layer below the target layer, the second feature fusion image of each target layer can be obtained.

[0091] Based on the above embodiments, as an optional embodiment, using an AG neural network, a second feature fusion image of the target layer is determined based on the encoded image of the target layer and the decoded image of the next layer below the target layer, including:

[0092] The encoded image of the target layer and the decoded image of the next layer of the target layer are input into the AG neural network to obtain the first feature fusion image of the target layer output by the AG neural network.

[0093] A feature concatenation operation is performed on the first feature fusion image of the target layer and the decoded image of the next layer of the target layer to obtain the second feature fusion image of the target layer. Specifically, in an embodiment of the present invention, the encoded image of the target layer and the decoded image of its next layer are input into an AG neural network to fuse the encoded and decoded information, thereby obtaining the first feature fusion image of the target layer output by the AG neural network.

[0094] Figure 2 This is a schematic diagram of the AG neural network structure in the deep learning-based knee joint image segmentation method provided by the present invention, as shown below. Figure 2 As shown, in an embodiment of the present invention, for the above steps, the encoded image of the target layer and the decoded image of the next layer of the target layer are input into the AG neural network to obtain the first feature fusion image of the target layer output by the AG neural network. The AG neural network is specifically used for:

[0095] The encoded image of the target layer and the decoded image of the next layer below the target layer are superimposed to obtain the seventh feature image;

[0096] Perform feature classification on the seventh feature image to obtain the second attention coefficient array;

[0097] The second attention coefficient array is multiplied by the encoded image of the target layer to obtain the eighth feature image;

[0098] The eighth feature image is subjected to three-dimensional convolution processing to obtain the first feature fusion image of the target layer.

[0099] Specifically, in the embodiments of the present invention, the input of the image AG neural network mainly consists of two parts: the encoded part (encoded image) output from the encoding layer and the decoded part (decoded image) output from the decoding layer. To ensure that the features of the encoding and decoding networks can be fused and calculated, an upsampling operation is performed on the decoding part. Therefore, it is necessary to upsample the decoded image of the layer following each target layer.

[0100] In an embodiment of the present invention, after the encoded image of the target layer and the upsampled decoded image of the next layer of the target layer are input into the AG neural network, the encoded image of the target layer and the upsampled decoded image of the next layer of the target layer will undergo an Add operation to add features, resulting in a new feature image, namely the seventh feature image. Furthermore, the seventh feature image will be subjected to a feature classification operation using the Softmax function to obtain the second attention coefficient array corresponding to the seventh feature image.

[0101] Furthermore, the second attention coefficient array is multiplied by the encoded image of the target layer to obtain the eighth feature image. The eighth feature image is then subjected to three-dimensional convolution, and the ninth feature image is sequentially processed by three-dimensional convolution Conv3×3 operation, BN operation and ReLU activation function operation to obtain the first feature fusion image of the target layer. Finally, the information fusion of the AG neural network is completed, realizing the feature fusion of the encoding and decoding information between the target layer and its next layer.

[0102] It should be noted that when there are multiple target layers, after the decoded images of each target layer are upsampled and the encoded images of the layer above them are input into the AG neural network, the encoding and decoding information of the decoded images of each target layer and the encoded images of the layer above them are fused in accordance with the above method.

[0103] The method in this embodiment of the invention adds an attention learning mechanism to the AG neural network and performs feature fusion on the encoding and decoding information between the two layers of the network. This allows the network to focus more on learning task-related features and suppress the learning of task-irrelevant parts, which is beneficial to improving the accuracy of subsequent knee joint image segmentation.

[0104] Furthermore, after obtaining the first feature fusion image of the target layer and the decoded image of the next layer of the target layer, the decoded image is upsampled, so that the first feature fusion image of the target layer and the upsampled decoded image of the next layer of the target layer can be concatted to obtain the second feature fusion image of the target layer.

[0105] The method of this invention performs feature fusion of encoding and decoding information between different layers of the knee joint image segmentation model network by deploying an AG neural network in the target layer, and obtains multi-scale feature fusion images of each target layer through image stitching operation, which provides reliable feature data for subsequent SCA module image segmentation and helps to improve the accuracy of knee joint image segmentation.

[0106] Furthermore, through the decoding network in the first layer, a feature concatenation operation can be performed on the encoded image of the first layer and the upsampled decoded image of the next target layer to obtain the decoded image of the first layer.

[0107] In embodiments of the present invention, the target layer may include a second layer, a third layer, and a fourth layer. After obtaining the decoded image of the second layer, the decoded image of the second layer can be upsampled, and a concat operation can be performed on the encoded image of the first layer and the upsampled decoded image of the second layer to obtain the decoded image of the first layer.

[0108] Furthermore, in an embodiment of the present invention, a feature concat operation is performed on the second feature fusion image of each target layer, the decoded image of the first layer, and the decoded image of the last layer to obtain a target feature concatenated image.

[0109] Finally, the target feature stitched image is input into the SCA module. Through the channel attention mechanism and spatial attention mechanism in the SCA module, multi-scale information fusion between the spatial and channel dimensions is performed on all image features contained in the target feature stitched image to obtain the knee joint segmentation image.

[0110] The method of this invention uses an AG neural network to perform feature fusion of encoding and decoding information in a knee joint image segmentation model, uses the Concat operation to perform multi-scale feature fusion of the decoding part, and uses the SCA module to perform multi-scale information fusion of all features output by each layer of the network in both spatial and channel dimensions. This effectively learns various detailed features of the knee joint image to be segmented, greatly improving the accuracy of knee joint image segmentation.

[0111] Figure 3 This is a schematic diagram of the network structure of the SCA module in the deep learning-based knee joint image segmentation method provided by this invention, as shown below. Figure 3 As shown, in the embodiments of the present invention, the SCA module mainly includes two parts: channel attention mechanism network 1 and spatial attention mechanism network 2.

[0112] In embodiments of the present invention, the SCA module is specifically used for:

[0113] Based on the channel attention mechanism, channel attention features are extracted from the target feature stitched image to obtain the target channel attention feature image;

[0114] Based on the spatial attention mechanism, spatial attention features are extracted from the target feature stitched image to obtain the target spatial attention feature image;

[0115] The target channel attention feature image and the target space attention feature image are superimposed to obtain the target feature fusion image. The target feature fusion image is then subjected to one-dimensional convolution to obtain the knee joint segmentation image.

[0116] Specifically, the target channel attention feature image described in this embodiment of the invention refers to an image containing important features of the knee joint obtained by extracting channel attention features from a target feature stitched image.

[0117] The target spatial attention feature image described in this embodiment of the invention refers to an image containing knee joint region features obtained by extracting spatial attention features from a target feature stitched image.

[0118] In embodiments of the present invention, the target feature fusion image refers to the image obtained by multi-scale information fusion of all image features contained in the target feature stitched image in both spatial and channel dimensions, and by feature fusion of the target channel attention feature image and the target spatial attention feature image.

[0119] Based on the content of the above embodiments, such as Figure 3 As shown, in an embodiment of the present invention, based on a channel attention mechanism, channel attention features are extracted from the target feature stitched image to obtain a target channel attention feature image, including:

[0120] The target feature stitched image is subjected to one-dimensional convolution to obtain the first feature image, and the target feature stitched image is subjected to three-dimensional convolution to obtain the second feature image.

[0121] The first feature image and the second feature image are superimposed to obtain the third feature image;

[0122] Perform feature classification on the third feature image to obtain the first attention coefficient array;

[0123] The first attention coefficient array is multiplied by the first feature image to obtain the target channel attention feature image.

[0124] Specifically, the one-dimensional convolution processing described in the embodiments of the present invention may include operations of one-dimensional convolution Conv1×1 operation, BN operation and ReLU activation function in sequence.

[0125] In an embodiment of the present invention, the target feature stitched image can first be processed by one-dimensional convolution through the channel attention mechanism of the SCA module, so that the target feature stitched image is sequentially processed by one-dimensional convolution Conv1×1 operation, BN operation and ReLU activation function operation to obtain the first feature image.

[0126] Simultaneously, the target feature stitched image can be processed by three-dimensional convolution, so that the target feature stitched image is sequentially processed by three-dimensional convolution Conv3×3 operation, BN operation and ReLU activation function to obtain the second feature image.

[0127] Furthermore, the first and second feature images are superimposed using an Add operation to obtain a new feature image, namely the third feature image. Then, the third feature image is subjected to feature classification using the Softmax function to obtain the first attention coefficient array corresponding to the third feature image.

[0128] Furthermore, the first attention coefficient array is multiplied by the first feature image to obtain a new feature image, which is the target channel attention feature image. Finally, the channel attention features are obtained, and the importance of different feature channels is learned.

[0129] The method of this invention, through the channel attention mechanism of the SCA module, can effectively learn the global information of the image feature channels, thereby selectively emphasizing useful important knee joint features and suppressing useless features, which is beneficial to improving the accuracy of subsequent image segmentation.

[0130] Furthermore, such as Figure 3As shown, in an embodiment of the present invention, spatial attention features are extracted from the target feature stitched image based on a spatial attention mechanism to obtain a target spatial attention feature image, including:

[0131] The first feature image is obtained by performing a one-dimensional convolution on the target feature stitched image.

[0132] Perform feature classification on the first feature image to obtain the fourth feature image;

[0133] Perform a dot product operation on the first feature image and the fourth feature image to obtain the fifth feature image;

[0134] Encode the fifth feature image to obtain the sixth feature image;

[0135] The first feature image and the sixth feature image are superimposed to obtain the target space attention feature image.

[0136] In embodiments of the present invention, the entire operation of the spatial attention mechanism of the SCA module is similar to the self-attention mechanism of the Transformer.

[0137] Specifically, in the embodiments of the present invention, through the spatial attention mechanism of the SCA module, the target feature stitched image can first be processed by one-dimensional convolution, so that the target feature stitched image is sequentially processed by one-dimensional convolution Conv1×1 operation, BN operation and ReLU activation function operation to obtain the first feature image, thereby obtaining 3 first feature images.

[0138] Then, the first feature image can be classified using the Softmax function to obtain the fourth feature image, and the second first feature image and the fourth feature image can be multiplied to obtain the fifth feature image.

[0139] In embodiments of the present invention, the encoding operation performed on the fifth feature image may specifically be a sequential operation of one-dimensional convolution, layer normalization (LN) and ReLU activation function.

[0140] Furthermore, the fifth feature image is encoded at different scales, so that the fifth feature image is encoded sequentially through one-dimensional convolution Conv1×1 operation, BN operation and ReLU activation function operation, LN operation and ReLU activation function operation to obtain the sixth feature image.

[0141] Furthermore, the third fourth feature image is superimposed on the sixth feature image using an Add operation to obtain the target spatial attention feature image, thereby finally obtaining the spatial attention features and completing the localization and processing of important feature areas of the knee joint.

[0142] The method of this invention calculates the feature importance of each pixel in the image at a spatial scale through the spatial attention mechanism of the SCA module, effectively locates the feature parts of important regions, and extracts feature information of key regions in the knee joint image, which helps to improve the accuracy of subsequent image segmentation.

[0143] Furthermore, in an embodiment of the present invention, after obtaining the target channel attention feature image and the target spatial attention feature image, the target channel attention feature image and the target spatial attention feature image can be superimposed by an Add operation. Combining the advantages of the channel attention mechanism and the spatial attention mechanism, multi-scale information is fused from both channel and spatial dimensions to obtain a target feature fusion image.

[0144] In an embodiment of the present invention, after obtaining the target feature fusion image, the target feature fusion image can be further processed by one-dimensional convolution, so that the target feature fusion image is sequentially processed by one-dimensional convolution Conv1×1 operation, BN operation and ReLU activation function operation to obtain the knee joint segmentation image.

[0145] The method of this invention combines the advantages of the channel attention mechanism and the spatial attention mechanism of the SCA module, and performs multi-scale information fusion on the feature images output by the decoding layer of each network from both channel and spatial dimensions. This allows for better learning of detailed features and is beneficial for high-precision image segmentation in cases where the knee joint has rich detailed features.

[0146] Figure 4 This is a schematic diagram of the network model structure in the deep learning-based knee joint image segmentation method provided by the present invention, as shown below. Figure 4 As shown, the multi-layer network may include five layers, namely the first layer, three target layers, and the last layer. It can be understood that the target layers include three layers, namely the second, third, and fourth layers of the multi-layer network. The specific steps of the deep learning-based knee joint image segmentation method in this embodiment of the invention may include:

[0147] Step 41: After inputting the knee joint image to be segmented into the knee joint image segmentation model, the knee joint image to be segmented is encoded through the encoding network of each layer in the five-layer network. The knee joint image to be segmented is encoded by performing three-dimensional convolution processing and MP operation to obtain the encoded images of each layer.

[0148] Step 42: Perform three-dimensional convolution processing again on the encoded image obtained after the last layer's three-dimensional convolution processing to obtain the decoded image of the last layer; and upsample the decoded image of the last layer. Input the encoded image of the fourth layer and the upsampled decoded image of the last layer into the AG neural network to obtain the first feature fusion image of the fourth layer output by the AG neural network; and perform a Concat operation on the first feature fusion image of the fourth layer and the upsampled decoded image of the last layer to obtain the decoded image of the fourth layer, which is also the second feature fusion image of the fourth layer.

[0149] Step 43: Upsample the decoded image of the fourth layer, and input the encoded image of the third layer and the upsampled decoded image of the fourth layer into the AG neural network to obtain the first feature fusion image of the third layer; and perform a Concat operation on the first feature fusion image of the third layer and the upsampled decoded image of the fourth layer to obtain the decoded image of the third layer, that is, to obtain the second feature fusion image of the third layer.

[0150] Step 44: Upsample the decoded image of the third layer, and input the encoded image of the second layer and the upsampled decoded image of the third layer into the AG neural network to obtain the first feature fusion image of the second layer; and perform a Concat operation on the first feature fusion image of the second layer and the upsampled decoded image of the third layer to obtain the decoded image of the second layer, that is, to obtain the second feature fusion image of the second layer.

[0151] Thus, through the above steps, the second feature fusion images of each target layer, namely the second, third and fourth layers, can be obtained.

[0152] Step 45: Upsample the decoded image of the second layer, and perform a Concat operation on the encoded image of the first layer and the upsampled decoded image of the second layer to obtain the decoded image of the first layer.

[0153] Further, in step 46, a Concat operation is performed on the decoded image of the first layer, the decoded image of the last layer, and the second feature fusion image of each target layer to obtain a target feature stitched image. This target feature stitched image is then input into the SCA module. Through the channel attention mechanism and spatial attention mechanism in the SCA module, multi-scale information fusion between the spatial and channel dimensions is performed on all image features contained in the target feature stitched image to obtain a target feature fusion image.

[0154] Finally, in step 47, the target feature fusion image is further processed by one-dimensional convolution, so that the target feature fusion image is sequentially processed by one-dimensional convolution Conv1×1 operation, BN operation and ReLU activation function to obtain the knee joint segmentation image.

[0155] In an embodiment of the present invention, by arranging three target layers in the network structure of the knee joint image segmentation model, using the AG neural network to perform feature fusion of the encoding and decoding information of each target layer, and using the SCA module to perform multi-scale information fusion again on the feature images output by the decoding layers of each network from both channel and spatial dimensions, this optimization setting can achieve high-quality image segmentation effect while also achieving high-efficiency image segmentation.

[0156] The following describes the deep learning-based knee joint image segmentation system provided by the present invention. The knee joint image segmentation system described below can be referred to in correspondence with the deep learning-based knee joint image segmentation method described above.

[0157] Figure 5 This is a schematic diagram of the deep learning-based knee joint image segmentation system provided by the present invention, as shown below. Figure 5 Shown, including:

[0158] The image segmentation module 510 is used to input the knee joint image to be segmented into the knee joint image segmentation model to obtain the knee joint segmentation image output by the knee joint image segmentation model.

[0159] The knee joint image segmentation model includes a multi-layer network and a spatial channel dual attention module. The multi-layer network includes at least one target layer, which is constructed based on an encoder-decoder network and an attention gate neural network.

[0160] The attention-gated neural network is used to fuse features between the encoded image of the target layer and the decoded image of the next layer below the target layer to generate the first feature fused image; the encoded image of the target layer is determined based on the knee joint image to be segmented.

[0161] The spatial channel dual attention module is used to fuse channel attention features and spatial attention features in the target feature stitched image to obtain a knee joint segmentation image; the target feature stitched image is determined based on the second feature fusion image output by each target layer; the second feature fusion image of the target layer is determined based on the first feature fusion image of the target layer and the decoded image of the next layer of the target layer.

[0162] The deep learning-based knee joint image segmentation system described in this embodiment can be used to execute the above-described deep learning-based knee joint image segmentation method embodiment. Its principle and technical effect are similar, and will not be repeated here.

[0163] The deep learning-based knee joint image segmentation system provided by this invention fuses the encoding and decoding information of the knee joint image to be segmented using an attention-gated neural network. Then, a spatial channel dual-attention module fuses multi-scale information across both spatial and channel dimensions for all features output from each network layer. This process constructs a knee joint image segmentation model using the encoding / decoding network, the attention-gated neural network, and the spatial channel dual-attention module. By inputting the knee joint image to be segmented into this model, the system can effectively learn various detailed features from the knee joint image, obtaining a high-precision segmented knee joint image. This system effectively completes image segmentation tasks, improving the accuracy of the segmentation results. Even in cases with rich knee joint details, it can achieve high-precision image segmentation results.

[0164] Based on the above embodiments, as an optional embodiment, the multilayer network sequentially includes a first layer, at least one target layer, and a last layer, wherein both the first layer and the last layer are encoding / decoding networks;

[0165] Knee joint image segmentation model, including:

[0166] The encoding module is used to encode the knee joint image to be segmented by sequentially passing it through the encoding networks of each layer in the multi-layer network, so as to obtain the encoded images of each layer of the multi-layer network;

[0167] The first decoding module is used to perform three-dimensional convolution processing on the encoded image of the last layer to obtain the decoded image of the last layer.

[0168] The feature fusion module is used to determine the second feature fusion image of the target layer based on the encoded image of the target layer and the decoded image of the next layer after the target layer through an attention-gated neural network.

[0169] The second decoding module is used to determine the decoded image of the first layer based on the encoded image of the first layer and the decoded image of the next target layer in the first layer through the decoding network in the first layer;

[0170] The feature stitching module is used to perform feature stitching operations on the second feature fusion image of each target layer, the decoded image of the first layer, and the decoded image of the last layer to obtain the target feature stitched image;

[0171] The image segmentation module is used to fuse channel attention features and spatial attention features of the target feature stitched image through the spatial channel dual attention module to obtain a knee joint segmentation image.

[0172] Based on the above embodiments, as an optional embodiment, the spatial channel dual attention module includes:

[0173] The first processing submodule is used to extract channel attention features from the target feature stitched image based on the channel attention mechanism to obtain the target channel attention feature image;

[0174] The second processing submodule is used to extract spatial attention features from the target feature stitched image based on the spatial attention mechanism to obtain the target spatial attention feature image;

[0175] The overlay submodule is used to overlay the target channel attention feature image and the target spatial attention feature image to obtain a target feature fusion image. Then, one-dimensional convolution processing is performed on the target feature fusion image to obtain a knee joint segmentation image.

[0176] Based on the above embodiments, as an optional embodiment, the first processing submodule is specifically used for:

[0177] The target feature stitched image is subjected to one-dimensional convolution to obtain the first feature image, and the target feature stitched image is subjected to three-dimensional convolution to obtain the second feature image.

[0178] The first feature image and the second feature image are superimposed to obtain the third feature image;

[0179] Perform feature classification on the third feature image to obtain the first attention coefficient array;

[0180] The first attention coefficient array is multiplied by the first feature image to obtain the target channel attention feature image.

[0181] Based on the above embodiments, as an optional embodiment, the second processing submodule is specifically used for:

[0182] The first feature image is obtained by performing a one-dimensional convolution on the target feature stitched image.

[0183] Perform feature classification on the first feature image to obtain the fourth feature image;

[0184] Perform a dot product operation on the first feature image and the fourth feature image to obtain the fifth feature image;

[0185] Encode the fifth feature image to obtain the sixth feature image;

[0186] The first feature image and the sixth feature image are superimposed to obtain the target space attention feature image.

[0187] Based on the above embodiments, as an optional embodiment, the feature fusion module is specifically used for:

[0188] The encoded image of the target layer and the decoded image of the next layer of the target layer are input into the attention gate neural network to obtain the first feature fusion image of the target layer output by the attention gate neural network.

[0189] The first feature fusion image of the target layer and the decoded image of the next layer of the target layer are subjected to feature concatenation to obtain the second feature fusion image of the target layer.

[0190] Based on the above embodiments, as an optional embodiment, the attention-gate neural network is specifically used for:

[0191] The encoded image of the target layer and the decoded image of the next layer below the target layer are superimposed to obtain the seventh feature image;

[0192] Perform feature classification on the seventh feature image to obtain the second attention coefficient array;

[0193] The second attention coefficient array is multiplied by the encoded image of the target layer to obtain the eighth feature image;

[0194] The eighth feature image is subjected to three-dimensional convolution processing to obtain the first feature fusion image of the target layer.

[0195] Figure 6 This is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as... Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute the deep learning-based knee joint image segmentation method provided by the above methods. This method includes: inputting a knee joint image to be segmented into a knee joint image segmentation model to obtain a segmented knee joint image output by the knee joint image segmentation model; the knee joint image segmentation model includes a multi-layer network and a spatial channel dual-attention module; the multi-layer network includes at least one target layer, which is constructed based on an encoding / decoding network and an attention-gated neural network; the attention-gated neural network is used to fuse features of the encoded image of the target layer and the decoded image of the next layer of the target layer to generate a first feature fusion image; the encoded image of the target layer is determined based on the knee joint image to be segmented; the spatial channel dual-attention module is used to fuse channel attention features and spatial attention features of the target feature stitched image to obtain the knee joint segmentation image; the target feature stitched image is determined based on the second feature fusion image output by each of the target layers; the second feature fusion image of the target layer is determined based on the first feature fusion image of the target layer and the decoded image of the next layer of the target layer.

[0196] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0197] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the deep learning-based knee joint image segmentation method provided by the above methods. The method includes: inputting a knee joint image to be segmented into a knee joint image segmentation model to obtain a knee joint segmentation image output by the knee joint image segmentation model; the knee joint image segmentation model includes a multi-layer network and a spatial channel dual attention module, the multi-layer network including at least one target layer, the target layer being constructed based on an encoding / decoding network and an attention gate neural network; the attention gate neural network being used to perform feature fusion on the encoded image of the target layer and the decoded image of the next layer of the target layer to generate a first feature fusion image; the encoded image of the target layer being determined based on the knee joint image to be segmented; the spatial channel dual attention module being used to fuse channel attention features and spatial attention features on the target feature stitched image to obtain the knee joint segmentation image; the target feature stitched image being determined based on the second feature fusion image output by each of the target layers; the second feature fusion image of the target layer being determined based on the first feature fusion image of the target layer and the decoded image of the next layer of the target layer.

[0198] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the deep learning-based knee joint image segmentation method provided by the above methods. This method includes: inputting a knee joint image to be segmented into a knee joint image segmentation model to obtain a knee joint segmentation image output by the knee joint image segmentation model; the knee joint image segmentation model includes a multi-layer network and a spatial channel dual-attention module, the multi-layer network including at least one target layer, the target layer being constructed based on an encoding / decoding network and an attention-gated neural network; the attention-gated neural network being used to perform feature fusion on the encoded image of the target layer and the decoded image of the next layer of the target layer to generate a first feature fusion image; the encoded image of the target layer being determined based on the knee joint image to be segmented; the spatial channel dual-attention module being used to fuse channel attention features and spatial attention features on the target feature stitched image to obtain the knee joint segmentation image; the target feature stitched image being determined based on the second feature fusion image output by each of the target layers; the second feature fusion image of the target layer being determined based on the first feature fusion image of the target layer and the decoded image of the next layer of the target layer.

[0199] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0200] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0201] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A knee joint image segmentation method based on deep learning, characterized in that, include: The knee joint image to be segmented is input into the knee joint image segmentation model to obtain the knee joint segmentation image output by the knee joint image segmentation model; The knee joint image segmentation model includes a multi-layer network and a spatial channel dual attention module. The multi-layer network includes at least one target layer, which is constructed based on an encoder-decoder network and an attention gate neural network. The attention-gated neural network is used to fuse features between the encoded image of the target layer and the decoded image of the next layer below the target layer to generate a first feature fusion image; the encoded image of the target layer is determined based on the knee joint image to be segmented. The spatial channel dual attention module is used to fuse channel attention features and spatial attention features in the target feature stitching image to obtain the knee joint segmentation image; the target feature stitching image is obtained by performing concat processing on the second feature fusion image output by each target layer, or by performing feature stitching concat operation on the second feature fusion image of each target layer, the decoded image of the first layer, and the decoded image of the last layer; the second feature fusion image of the target layer is determined based on the first feature fusion image of the target layer and the decoded image of the next layer of the target layer.

2. The deep learning-based knee joint image segmentation method according to claim 1, characterized in that, The multi-layer network sequentially includes a first layer, the at least one target layer, and a last layer, wherein the first layer and the last layer are both encoding / decoding networks; The knee joint image segmentation model is specifically used for: The knee joint image to be segmented is encoded sequentially through the encoding networks of each layer in the multi-layer network to obtain the encoded images of each layer in the multi-layer network; The encoded image of the last layer is subjected to three-dimensional convolution processing to obtain the decoded image of the last layer; The second feature fusion image of the target layer is determined using the attention-gated neural network based on the encoded image of the target layer and the decoded image of the next layer after the target layer. The decoded image of the first layer is determined by the decoding network in the first layer, based on the encoded image of the first layer and the decoded image of the next target layer. A feature stitching operation is performed on the second feature fusion image of each target layer, the decoded image of the first layer, and the decoded image of the last layer to obtain the target feature stitched image; The spatial channel dual attention module is used to fuse channel attention features and spatial attention features in the target feature stitched image to obtain the knee joint segmentation image.

3. The deep learning-based knee joint image segmentation method according to claim 2, characterized in that, The spatial channel dual attention module is specifically used for: Based on the channel attention mechanism, channel attention features are extracted from the target feature stitched image to obtain the target channel attention feature image; Based on the spatial attention mechanism, spatial attention features are extracted from the target feature stitched image to obtain the target spatial attention feature image; The target channel attention feature image and the target spatial attention feature image are superimposed to obtain a target feature fusion image. The target feature fusion image is then subjected to one-dimensional convolution to obtain the knee joint segmentation image.

4. The deep learning-based knee joint image segmentation method according to claim 3, characterized in that, The method based on channel attention mechanism, which extracts channel attention features from the target feature stitched image to obtain a target channel attention feature image, includes: The target feature stitched image is subjected to one-dimensional convolution processing to obtain a first feature image, and the target feature stitched image is subjected to three-dimensional convolution processing to obtain a second feature image; The first feature image and the second feature image are superimposed to obtain the third feature image; Perform feature classification on the third feature image to obtain a first attention coefficient array; The first attention coefficient array is multiplied by the first feature image to obtain the target channel attention feature image.

5. The deep learning-based knee joint image segmentation method according to claim 3, characterized in that, The spatial attention mechanism-based extraction of spatial attention features from the target feature stitched image to obtain a target spatial attention feature image includes: The target feature stitched image is subjected to one-dimensional convolution processing to obtain the first feature image; Perform feature classification on the first feature image to obtain the fourth feature image; Perform a dot product operation on the first feature image and the fourth feature image to obtain the fifth feature image; The fifth feature image is encoded to obtain the sixth feature image; The first feature image and the sixth feature image are superimposed to obtain the target space attention feature image.

6. The deep learning-based knee joint image segmentation method according to claim 2, characterized in that, The step of determining the second feature fusion image of the target layer based on the encoded image of the target layer and the decoded image of the next layer after the target layer through the attention-gated neural network includes: The encoded image of the target layer and the decoded image of the next layer of the target layer are input into the attention gate neural network to obtain the first feature fusion image of the target layer output by the attention gate neural network; A feature concatenation operation is performed on the first feature fusion image of the target layer and the decoded image of the next layer of the target layer to obtain the second feature fusion image of the target layer.

7. The deep learning-based knee joint image segmentation method according to claim 6, characterized in that, The attention-gated neural network is specifically used for: The encoded image of the target layer and the decoded image of the next layer below the target layer are superimposed to obtain the seventh feature image; Perform feature classification on the seventh feature image to obtain a second attention coefficient array; The second attention coefficient array is multiplied by the encoded image of the target layer to obtain the eighth feature image. The eighth feature image is subjected to three-dimensional convolution processing to obtain the first feature fusion image of the target layer.

8. A knee joint image segmentation system based on deep learning, characterized in that, include: The image segmentation module is used to input the knee joint image to be segmented into the knee joint image segmentation model to obtain the knee joint segmented image output by the knee joint image segmentation model; The knee joint image segmentation model includes a multi-layer network and a spatial channel dual attention module. The multi-layer network includes at least one target layer, which is constructed based on an encoder-decoder network and an attention gate neural network. The attention-gated neural network is used to fuse features between the encoded image of the target layer and the decoded image of the next layer below the target layer to generate a first feature fusion image; the encoded image of the target layer is determined based on the knee joint image to be segmented. The spatial channel dual attention module is used to fuse channel attention features and spatial attention features in the target feature stitching image to obtain the knee joint segmentation image; the target feature stitching image is obtained by performing concat processing on the second feature fusion image output by each target layer, or by performing feature stitching concat operation on the second feature fusion image of each target layer, the decoded image of the first layer, and the decoded image of the last layer; the second feature fusion image of the target layer is determined based on the first feature fusion image of the target layer and the decoded image of the next layer of the target layer.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the deep learning-based knee joint image segmentation method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the deep learning-based knee joint image segmentation method as described in any one of claims 1 to 7.

Images