Multi-organ identification method and device based on CNN structure and Transform structure
By combining the multi-organ recognition method with CNN and Transformer structures, the problem of low organ recognition accuracy is solved, especially in CT medical images, and the recognition accuracy is improved by extracting local features and capturing long-range dependencies.
Patent Information
- Application Number
- CN202411734041.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-03-21
AI Technical Summary
The accuracy of organ recognition is low, especially in CT medical images, which are disturbances due to patient movement or image noise.
Using a multi-organ recognition method based on CNN structure and Transformer structure, multi-level local features of the image are extracted through CNN, Transformer captures the long-range dependence of different sub-blocks of the image, and improves the recognition accuracy through interactive modules.
It improves the accuracy of multi-organ recognition and effectively makes up for the problem of insufficient accuracy caused by CT image interference.
Smart Images
Figure CN119671973A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of medical image processing technology, and more specifically, to a multi-organ recognition method and device based on CNN structure and Transformer structure. Background Art
[0002] In the process of organ recognition, CT medical images with high resolution and clarity are required. However, in some cases, such as patient movement during scanning and image noise, the accuracy of organ recognition in CT medical images will decrease. Summary of the invention
[0003] The problem solved by this application is the low accuracy of organ recognition.
[0004] In order to solve the above problems, the first aspect of the present application provides a multi-organ recognition method based on CNN structure and Transformer structure, which includes:
[0005] Acquire a combined chest and abdomen medical image of any subject;
[0006] Preprocessing the chest-abdomen combined medical image;
[0007] The processed chest and abdomen combined medical image is input into a multi-organ recognition model to obtain an organ recognition result; the multi-organ recognition model is composed of a CNN structure branch and a Transformer structure branch, and an interactive module that performs interactive processing on the CNN structure branch and the Transformer structure branch.
[0008] The second aspect of the present application provides a manufacturing system for a multi-organ recognition method based on a CNN structure and a Transformer structure, which comprises:
[0009] An image acquisition module, which is used to acquire a chest and abdomen combined medical image of any object;
[0010] A preprocessing module, which is used to preprocess the chest and abdomen combined medical image;
[0011] The organ recognition module is used to input the processed chest and abdomen combined medical image into a multi-organ recognition model to obtain an organ recognition result; the multi-organ recognition model is composed of a CNN structure branch and a Transformer structure branch, and an interactive module that performs interactive processing on the CNN structure branch and the Transformer structure branch.
[0012] The third aspect of the present application provides an electronic device, comprising: a memory and a processor; the memory may be configured to store a program, and the processor is coupled to the memory and is used to execute the program in the memory, so as to:
[0013] Acquire a combined chest and abdomen medical image of any subject;
[0014] Preprocessing the chest-abdomen combined medical image;
[0015] The processed chest and abdomen combined medical image is input into a multi-organ recognition model to obtain an organ recognition result; the multi-organ recognition model is composed of a CNN structure branch and a Transformer structure branch, and an interactive module that performs interactive processing on the CNN structure branch and the Transformer structure branch.
[0016] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the aforementioned multi-organ recognition method based on CNN structure and Transformer structure.
[0017] In this application, the CNN structure is used to extract multi-level local features of the input image; the Transformer structure is used to capture the long-range dependencies of different sub-blocks of the image. The interaction between the two improves the accuracy of multi-organ recognition and compensates for the lack of accuracy caused by interference from CT images. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 It is a flowchart of a multi-organ recognition method based on a CNN structure and a Transformer structure according to an embodiment of the present application;
[0019] Figure 2 It is an architecture diagram of a network model of a multi-organ recognition method based on a CNN structure and a Transformer structure according to an embodiment of the present application;
[0020] Figure 3 It is an architecture diagram of an interaction module of a multi-organ recognition method based on a CNN structure and a Transformer structure according to an embodiment of the present application;
[0021] Figure 4 It is an architecture diagram of a multi-organ recognition device based on a CNN structure and a Transformer structure according to an embodiment of the present application;
[0022] Figure 5 4 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0023] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described in detail below in conjunction with the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.
[0024] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by technicians in the field to which this application belongs.
[0025] The embodiment of the present application provides the above-mentioned multi-organ recognition method based on CNN structure and Transformer structure, and the specific scheme of the method is as follows: Figure 1-Figure 3 As shown, the method can be performed by a multi-organ recognition device based on a CNN structure and a Transformer structure, and the multi-organ recognition device based on a CNN structure and a Transformer structure can be integrated into electronic devices such as computers, servers, computers, server clusters, and data centers. Figure 1 As shown, the multi-organ recognition method based on CNN structure and Transformer structure includes:
[0026] S101, acquiring a chest and abdomen combined medical image of any subject;
[0027] In the present application, the medical image is a medical CT image (Computed Tomography, CT).
[0028] S102, preprocessing the chest and abdomen combined medical image;
[0029] S103, inputting the processed chest and abdomen combined medical image into a multi-organ recognition model to obtain an organ recognition result; the multi-organ recognition model is composed of a CNN structure branch and a Transformer structure branch, and an interactive module that performs interactive processing on the CNN structure branch and the Transformer structure branch.
[0030] In this application, the CNN structure is used to extract multi-level local features of the input image; the Transformer structure is used to capture the long-range dependencies of different sub-blocks of the image. The interaction between the two improves the accuracy of multi-organ recognition and compensates for the lack of accuracy caused by interference from CT images.
[0031] The multiple organs include: left lung, right lung, liver, gallbladder, spleen, left kidney, right kidney, and inferior vena cava.
[0032] In a specific implementation, the step S103, inputting the processed chest and abdomen combined medical image into a multi-organ recognition model to obtain an organ recognition result, includes:
[0033] The processed chest and abdomen joint medical images are input into the CNN structure and the Transformer structure respectively to obtain the corresponding first feature map and second feature map;
[0034] Inputting the first feature graph and the second feature graph into an interaction module for interaction processing to obtain a first interaction graph and a second interaction graph after interaction;
[0035] The first interaction graph is sequentially input into the CNN structure and the dynamic extraction module to obtain a dynamic extraction graph;
[0036] The second interaction graph is sequentially input into the Transformer structure and the multi-convolution module to obtain a multi-convolution feature graph;
[0037] After adding the dynamic extraction map and the multi-convolution feature map, convolution processing is performed to obtain the organ recognition result.
[0038] In this application, the CNN structure is used to extract multi-level local features of the input image; the Transformer structure is used to capture the long-range dependencies of different sub-blocks of the image. Through the interactive module, the outputs of the CNN structure and the Transformer structure are cross-interacted and bound to each other, thereby making full use of the local context features from the CNN and the global context information from the SwinTransformer.
[0039] In this application, the CNN structure and the Transformer structure are conventional structures, and their specific structural details will not be repeated in this application.
[0040] In this application, a multi-convolution module refers to multiple convolutions arranged continuously.
[0041] In the present application, the convolution processing performed after the addition may be a single convolution processing or multiple consecutive convolutions.
[0042] In a specific implementation, the preprocessing is a deblurring process, a denoising process and an image enhancement process arranged in sequence.
[0043] In a specific embodiment, combining Figure 2 As shown, the processed chest and abdomen joint medical image is input into the CNN structure and the Transformer structure respectively to obtain the corresponding first feature map and second feature map, including:
[0044] Generate a coronal medical image and a sagittal medical image based on the processed chest-abdomen combined medical image;
[0045] Inputting the coronal medical image into a CNN structure to obtain a first feature map;
[0046] The sagittal medical image is input into the Transformer structure and transposed to obtain a second feature map.
[0047] In a specific embodiment, combining Figure 3 As shown, the first feature map and the second feature map are input into the interaction module for interaction processing to obtain the first interaction map and the second interaction map after interaction, including:
[0048] After rearranging the second feature map, the second feature map is spliced with the first feature map to obtain a first spliced map;
[0049] Performing multi-layer perception processing on the first spliced image to obtain corresponding first weights and second weights;
[0050] The result of multiplying the first weight and the first feature map and the result of multiplying the second weight and the rearranged second feature map are concatenated to obtain a second concatenated map;
[0051] After performing pooling and convolution processing on different branches of the second spliced image, the second spliced image is spliced again to obtain a third spliced image;
[0052] The third concatenated image is input into the convolution module, and after activation processing, it is multiplied with the first feature map and the rearranged second feature map respectively to obtain the corresponding first interaction map and second interaction map.
[0053] In the present application, rearrangement is used to adjust the dimension of the multi-dimensional data so that the second feature map has the same size as the first feature map.
[0054] In the present application, the features in the first feature map and the features in the second feature map are embedded into two spatial weights (first weight and second weight) through multi-layer perception processing.
[0055] Among them, MLP (multi-layer perceptual processing) is usually composed of multiple layers (including input layer, hidden layer and output layer), each of which contains multiple neurons, and the neurons are connected by weights.
[0056] Preferably, after multi-layer perception processing, 1×1 convolution, batch normalization and ReLU activation function may also be included to achieve complete embedding of features.
[0057] In the present application, the result of multiplying the first weight by the first feature map and the result of multiplying the second weight by the rearranged second feature map are spliced to obtain a second spliced map; specifically: multiplying the first weight by the first feature map to obtain result 1, multiplying the second weight by the rearranged second feature map to obtain result 2, and splicing result 1 and result 2 to obtain a second spliced map.
[0058] In the present application, the second spliced image is subjected to pooling and convolution processing of different branches respectively, and then spliced again to obtain a third spliced image; specifically: the second spliced image is subjected to global maximum pooling and convolution processing to obtain result 1; the second spliced image is subjected to global average pooling and convolution processing to obtain result 2; result 1 and result 2 are spliced to obtain a third spliced image.
[0059] In this application, average pooling and maximum pooling are used through different branches to retain more features of the first feature map and the second feature map.
[0060] Preferably, after average pooling / maximum pooling, attention vector extraction is further performed to achieve maximum retention of the first feature map and the second feature map.
[0061] In this application, the convolution module includes convolution, batch normalization and ReLU activation function processing.
[0062] In the present application, the third spliced image is input into the convolution module, and after the activation processing, it is multiplied with the first feature map and the rearranged second feature map respectively to obtain the corresponding first interaction map and the second interaction map. Specifically, the third spliced image is input into the convolution module, and after the Sigmoid activation processing, the weight value 1 is obtained, and the weight value 1 is multiplied by the first feature map to obtain the corresponding first interaction map; the weight value 1 is complementary to the weight value 1 as the weight value 2 (the sum of the two complementary values is 1), and the weight value 2 is multiplied by the second feature map to obtain the corresponding second interaction map.
[0063] In a specific implementation, the processing process of the dynamic extraction module is:
[0064] The input feature map is processed by average pooling, full connection, Relu activation, full connection, and Softmax function to generate complementary first weights and second weights;
[0065] Perform spatial attention extraction on the input feature map and multiply it by the first weight to obtain a first extraction map;
[0066] Perform channel attention extraction on the input feature map and multiply it by the second weight to obtain a second extraction map;
[0067] After adding the first extraction map and the second extraction map, an output feature map is obtained.
[0068] In this application, channel attention extraction aims to make the network focus on different channels of the input feature map, thereby enhancing the weights of important feature channels and weakening unimportant channels.
[0069] In this application, spatial attention extraction focuses on different positions of the input feature map in the spatial dimension, thereby guiding the network to pay more attention to important spatial positions in the image.
[0070] In this application, channel attention and spatial attention are combined. Spatial attention generates attention maps in height and width dimensions to enhance attention to specific spatial areas.
[0071] In this application, the normalized attention weights of the two branches are generated through average pooling, full connection, Relu activation, full connection, and Softmax function processing, and the global average pool is used to increase the receptive field, which allows the above processing to capture features from the entire image.
[0072] In this application, if the input features change, the weights of the two branches will also change; by limiting the dynamic weights, the learning of the entire dynamic extraction is promoted, and dynamic attention adjustment is achieved.
[0073] In this application, some unimportant attention features can be eliminated through dynamic extraction to achieve a balance between the spatial attention branch and the channel attention branch.
[0074] An embodiment of the present application provides a multi-organ recognition device based on a CNN structure and a Transformer structure, which is used to execute the multi-organ recognition method based on a CNN structure and a Transformer structure described in the above content of the present application. The multi-organ recognition device based on a CNN structure and a Transformer structure is described in detail below.
[0075] like Figure 4 As shown, the multi-organ recognition device based on CNN structure and Transformer structure includes:
[0076] An image acquisition module 101 is used to acquire a chest and abdomen combined medical image of any object;
[0077] A preprocessing module 102, which is used to preprocess the chest-abdomen combined medical image;
[0078] The organ recognition module 103 is used to input the processed chest and abdomen combined medical image into a multi-organ recognition model to obtain an organ recognition result; the multi-organ recognition model is composed of a CNN structure branch and a Transformer structure branch, and an interactive module that performs interactive processing on the CNN structure branch and the Transformer structure branch.
[0079] In one embodiment, the multiple organs include: left lung, right lung, liver, gallbladder, spleen, left kidney, right kidney, and inferior vena cava.
[0080] In one embodiment, the organ identification module 103 is further configured to:
[0081] The processed chest and abdomen joint medical image is respectively input into the CNN structure and the Transformer structure to obtain the corresponding first feature map and the second feature map; the first feature map and the second feature map are input into the interactive module for interactive processing to obtain the first interaction map and the second interaction map after interaction; the first interaction map is successively input into the CNN structure and the dynamic extraction module to obtain the dynamic extraction map; the second interaction map is successively input into the Transformer structure and the multi-convolution module to obtain the multi-convolution feature map; after adding the dynamic extraction map and the multi-convolution feature map, a convolution process is performed to obtain the organ recognition result.
[0082] In one implementation, the preprocessing is a deblurring process, a denoising process, and an image enhancement process that are arranged in sequence.
[0083] In one embodiment, the organ identification module 103 is further configured to:
[0084] A coronal medical image and a sagittal medical image are generated based on the processed chest-abdomen joint medical image; the coronal medical image is input into the CNN structure to obtain a first feature map; the sagittal medical image is input into the Transformer structure and transposed to obtain a second feature map.
[0085] In one embodiment, the organ identification module 103 is further configured to:
[0086] After the second feature map is rearranged, it is spliced with the first feature map to obtain a first spliced map; multi-layer perception processing is performed on the first spliced map to obtain corresponding first weights and second weights; the result of multiplying the first weight by the first feature map and the result of multiplying the second weight by the rearranged second feature map are spliced to obtain a second spliced map; the second spliced map is subjected to pooling and convolution processing of different branches respectively, and then spliced again to obtain a third spliced map; the third spliced map is input into the convolution module, and after activation processing, it is multiplied with the first feature map and the rearranged second feature map respectively to obtain the corresponding first interaction map and second interaction map.
[0087] In one embodiment, the organ identification module 103 is further configured to:
[0088] The input feature map is processed by average pooling, full connection, Relu activation, full connection, and Softmax function to generate complementary first weights and second weights; spatial attention extraction is performed on the input feature map and multiplied with the first weight to obtain a first extraction map; channel attention extraction is performed on the input feature map and multiplied with the second weight to obtain a second extraction map; the first extraction map and the second extraction map are added to obtain the output feature map.
[0089] The multi-organ recognition device based on CNN structure and Transformer structure provided in the above-mentioned embodiment of the present application has a corresponding relationship with the multi-organ recognition method based on CNN structure and Transformer structure provided in the embodiment of the present application. Therefore, the specific content in the system has a corresponding relationship with the multi-organ recognition method based on CNN structure and Transformer structure. The specific content can refer to the record in the multi-organ recognition method based on CNN structure and Transformer structure, and will not be repeated in this application.
[0090] The multi-organ recognition device based on CNN structure and Transformer structure provided in the above-mentioned embodiments of the present application and the multi-organ recognition method based on CNN structure and Transformer structure provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.
[0091] The above describes the internal functions and structures of the multi-organ recognition device based on the CNN structure and the Transformer structure. Figure 5 As shown, in practice, the multi-organ recognition device based on CNN structure and Transformer structure can be implemented as an electronic device, including: a memory 301 and a processor 303.
[0092] The memory 301 may be configured to store programs.
[0093] In addition, the memory 301 may also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.
[0094] The memory 301 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The processor 303, coupled to the memory 301, is used to execute the program in the memory 301, so as to:
[0095] Acquire a combined chest and abdomen medical image of any subject;
[0096] Preprocessing the chest-abdomen combined medical image;
[0097] The processed chest and abdomen combined medical image is input into a multi-organ recognition model to obtain an organ recognition result; the multi-organ recognition model is composed of a CNN structure branch and a Transformer structure branch, and an interactive module that performs interactive processing on the CNN structure branch and the Transformer structure branch.
[0098] In one embodiment, the multiple organs include: left lung, right lung, liver, gallbladder, spleen, left kidney, right kidney, and inferior vena cava.
[0099] In one implementation, the processor 303 is further configured to:
[0100] The processed chest and abdomen joint medical image is respectively input into the CNN structure and the Transformer structure to obtain the corresponding first feature map and the second feature map; the first feature map and the second feature map are input into the interactive module for interactive processing to obtain the first interaction map and the second interaction map after interaction; the first interaction map is successively input into the CNN structure and the dynamic extraction module to obtain the dynamic extraction map; the second interaction map is successively input into the Transformer structure and the multi-convolution module to obtain the multi-convolution feature map; after adding the dynamic extraction map and the multi-convolution feature map, a convolution process is performed to obtain the organ recognition result.
[0101] In one implementation, the preprocessing is a deblurring process, a denoising process, and an image enhancement process that are arranged in sequence.
[0102] In one implementation, the processor 303 is further configured to:
[0103] A coronal medical image and a sagittal medical image are generated based on the processed chest-abdomen joint medical image; the coronal medical image is input into the CNN structure to obtain a first feature map; the sagittal medical image is input into the Transformer structure and transposed to obtain a second feature map.
[0104] In one implementation, the processor 303 is further configured to:
[0105] After the second feature map is rearranged, it is spliced with the first feature map to obtain a first spliced map; multi-layer perception processing is performed on the first spliced map to obtain corresponding first weights and second weights; the result of multiplying the first weight by the first feature map and the result of multiplying the second weight by the rearranged second feature map are spliced to obtain a second spliced map; the second spliced map is subjected to pooling and convolution processing of different branches respectively, and then spliced again to obtain a third spliced map; the third spliced map is input into the convolution module, and after activation processing, it is multiplied with the first feature map and the rearranged second feature map respectively to obtain the corresponding first interaction map and second interaction map.
[0106] In one implementation, the processor 303 is further configured to:
[0107] The input feature map is processed by average pooling, full connection, Relu activation, full connection, and Softmax function to generate complementary first weights and second weights; spatial attention extraction is performed on the input feature map and multiplied with the first weight to obtain a first extraction map; channel attention extraction is performed on the input feature map and multiplied with the second weight to obtain a second extraction map; the first extraction map and the second extraction map are added to obtain the output feature map.
[0108] In this application, Figure 5 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 5 Components shown.
[0109] The electronic device provided in this embodiment is based on the same inventive concept as the multi-organ recognition method based on CNN structure and Transformer structure provided in the embodiment of the present application, and has the same beneficial effects as the method adopted, run or implemented by the application stored in it.
[0110] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0111] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0112] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0113] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash memory (Flash RAM). The memory is an example of a computer-readable medium.
[0114] The present application also provides a computer-readable storage medium corresponding to the multi-organ recognition method based on CNN structure and Transformer structure provided in the aforementioned embodiment, on which a computer program (i.e., program product) is stored. When the computer program is run by a processor, it will execute the multi-organ recognition method based on CNN structure and Transformer structure provided in any of the aforementioned embodiments.
[0115] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0116] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the multi-organ recognition method based on CNN structure and Transformer structure provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.
[0117] It should be noted that in the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.
[0118] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0119] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A multi-organ recognition method based on CNN structure and Transformer structure, characterized in that: include: Acquire a combined chest and abdomen medical image of any subject; Preprocessing the chest-abdomen combined medical image; The processed chest and abdomen combined medical images are input into the multi-organ recognition model to obtain the organ recognition results; The multi-organ recognition model consists of a CNN structure branch and a Transformer structure branch, and an interactive module that performs interactive processing on the CNN structure branch and the Transformer structure branch.
2. The multi-organ recognition method based on CNN structure and Transformer structure according to claim 1 is characterized in that: The multiple organs include: left lung, right lung, liver, gallbladder, spleen, left kidney, right kidney, and inferior vena cava.
3. The multi-organ recognition method based on CNN structure and Transformer structure according to claim 1 is characterized in that: The processed chest and abdomen combined medical images are input into the multi-organ recognition model to obtain organ recognition results, including: The processed chest and abdomen joint medical images are input into the CNN structure and the Transformer structure respectively to obtain the corresponding first feature map and second feature map; Inputting the first feature graph and the second feature graph into an interaction module for interaction processing to obtain a first interaction graph and a second interaction graph after interaction; The first interaction graph is sequentially input into the CNN structure and the dynamic extraction module to obtain a dynamic extraction graph; The second interaction graph is sequentially input into the Transformer structure and the multi-convolution module to obtain a multi-convolution feature graph; After adding the dynamic extraction map and the multi-convolution feature map, convolution processing is performed to obtain the organ recognition result.
4. The multi-organ recognition method based on CNN structure and Transformer structure according to any one of claims 1 to 3, characterized in that: The preprocessing is a deblurring process, a denoising process and an image enhancement process which are arranged in sequence.
5. The multi-organ recognition method based on CNN structure and Transformer structure according to claim 3 is characterized in that: The processed chest and abdomen joint medical images are input into the CNN structure and the Transformer structure respectively to obtain the corresponding first feature map and second feature map, including: Generate a coronal medical image and a sagittal medical image based on the processed chest-abdomen combined medical image; Inputting the coronal medical image into a CNN structure to obtain a first feature map; The sagittal medical image is input into the Transformer structure and transposed to obtain a second feature map.
6. The multi-organ recognition method based on CNN structure and Transformer structure according to claim 3 is characterized in that: The step of inputting the first feature graph and the second feature graph into the interaction module for interaction processing to obtain the first interaction graph and the second interaction graph after interaction includes: After rearranging the second feature map, the second feature map is spliced with the first feature map to obtain a first spliced map; Performing multi-layer perception processing on the first spliced image to obtain corresponding first weights and second weights; The result of multiplying the first weight and the first feature map and the result of multiplying the second weight and the rearranged second feature map are concatenated to obtain a second concatenated map; After performing pooling and convolution processing on different branches of the second spliced image, the second spliced image is spliced again to obtain a third spliced image; The third concatenated image is input into the convolution module, and after activation processing, it is multiplied with the first feature map and the rearranged second feature map respectively to obtain the corresponding first interaction map and second interaction map.
7. The multi-organ recognition method based on CNN structure and Transformer structure according to claim 3 is characterized in that: The processing process of the dynamic extraction module is: The input feature map is processed by average pooling, full connection, Relu activation, full connection, and Softmax function to generate complementary first weights and second weights; Perform spatial attention extraction on the input feature map and multiply it by the first weight to obtain a first extraction map; Perform channel attention extraction on the input feature map and multiply it by the second weight to obtain a second extraction map; After adding the first extraction map and the second extraction map, an output feature map is obtained.
8. A multi-organ recognition device based on CNN structure and Transformer structure, characterized in that: include: An image acquisition module, which is used to acquire a chest and abdomen combined medical image of any object; A preprocessing module, which is used to preprocess the chest and abdomen combined medical image; The organ recognition module is used to input the processed chest and abdomen combined medical image into a multi-organ recognition model to obtain an organ recognition result; the multi-organ recognition model is composed of a CNN structure branch and a Transformer structure branch, and an interactive module that performs interactive processing on the CNN structure branch and the Transformer structure branch.
9. An electronic device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor, coupled to the memory, is configured to execute the program to: Acquire a combined chest and abdomen medical image of any subject; Preprocessing the chest-abdomen combined medical image; The processed chest and abdomen combined medical images are input into the multi-organ recognition model to obtain the organ recognition results; The multi-organ recognition model consists of a CNN structure branch and a Transformer structure branch, and an interactive module that performs interactive processing on the CNN structure branch and the Transformer structure branch.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the multi-organ recognition method based on CNN structure and Transformer structure as described in any one of claims 1-6.