An enhancement method for pathological examination medical images

Abstract

The application relates to an enhancement method for pathological examination medical images. In the implementation, an initial medical image is converted into a first gray image, so as to facilitate subsequent image feature transformation; then a first feature map of multiple scales is extracted from the first gray image; then a first feature map of a higher scale and a first feature map of the lowest scale are selected, the first feature map of the higher scale is used to enhance edges, textures and other features in the first feature map of the lowest scale, so that the first feature map of the lowest scale corresponds to a second feature map of the lowest scale after enhancement; then a second module fused with a convolution operation and an attention mechanism is used to output a third feature map of multiple scales with global and partial feature enhancement; finally, the above feature maps are fused into a second fusion feature map, a second gray image is reconstructed, and finally an enhanced medical image is converted. The method can improve the enhancement effect of the medical image.

Description

Technical Field

[0001] This application relates to the field of audio recommendation technology, and more particularly to a method for enhancing medical images used in pathological examination. Background Technology

[0002] CT imaging technology uses X-rays to penetrate the human body and obtains tomographic images through computer processing, providing high-resolution anatomical information. CT images have unique advantages in displaying bones and certain tissue structures. In clinical practice, doctors often need to identify tiny lesions and structural details from CT images; however, due to problems such as noise, artifacts, and insufficient contrast, these details are often difficult to identify.

[0003] Therefore, in order to improve the structural details of CT images, neural networks are currently widely used to enhance CT images. A common method is to use CNN (Convolutional Neural Network) enhancement, but it still has the following drawbacks:

[0004] The shallow features extracted by CNNs contain rich edge and non-edge details. However, CNNs rely too much on learning deep features, which leads to insufficient extraction of these details and poor CT image enhancement. Moreover, the limitations of convolutional operations in terms of local receptive fields may pose challenges in capturing global features, resulting in poor CT image enhancement. Summary of the Invention

[0005] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0006] The main objective of this invention is to provide a method, system, device, and medium for enhancing medical images used in pathological examinations.

[0007] To achieve the above objectives, a first aspect of the present invention provides a method for enhancing medical images used in pathological examinations, the method comprising:

[0008] Frames are extracted from medical images to obtain an initial medical image, and the initial medical image is converted into a first grayscale image;

[0009] Extract a multi-scale first feature map from the first grayscale image;

[0010] The lowest-scale first feature map and the higher-scale first feature map in the multi-scale first feature map are input into the first module, so as to enhance the edge texture features in the lowest-scale first feature map according to the first module and based on the higher-scale first feature map, to obtain the lowest-scale second feature map corresponding to the lowest-scale first feature map.

[0011] The first feature map of the multi-scale is input into the second module, so that the third feature map of the multi-scale corresponding to the first feature map of the multi-scale is output according to the second module's fusion of convolution and attention mechanisms.

[0012] The first fused feature map is obtained by fusing the third feature map of the multi-scale, and the first fused feature map is fused with the second feature map of the lowest scale and the feature map of the first grayscale image to obtain the second fused feature map. The second fused feature map is then reconstructed to obtain the second grayscale image.

[0013] The second grayscale image is converted into the final medical image.

[0014] The enhancement method for pathological examination medical images provided in this application has at least the following beneficial effects:

[0015] To fully extract the rich edge and non-edge texture details from the lowest-scale first feature map, this method first inputs the lowest-scale and higher-scale first feature maps into the first module. The first module then utilizes the edge and non-edge texture details from the higher-scale first feature map to facilitate the learning of edge and non-edge details in the shallow features. Next, by fusing convolution and attention mechanisms, the global and local features of the lesion structure are effectively learned. Finally, the images are fused, which overall improves the enhancement effect on medical images.

[0016] In some embodiments, extracting a multi-scale first feature map from the first grayscale image includes:

[0017] Configure multiple Transformer blocks and cascade them; wherein the multiple Transformer blocks include at least four;

[0018] The first grayscale image is input into the plurality of Transformer blocks to obtain the multi-scale first feature map output by the plurality of Transformer blocks.

[0019] In some embodiments, obtaining a second feature map of the lowest scale corresponding to the lowest scale first feature map by enhancing the edge texture features in the lowest scale first feature map based on the first module and the higher-scale first feature map includes:

[0020] According to the first module, two high-scale first feature maps and one low-scale first feature map are selected in descending order of scale.

[0021] The first module performs the following operations on the two high-scale first feature maps:

[0022] ;

[0023] in, The first feature map at the higher scale is one of the two higher-scale first feature maps. The first feature map at the lower scale is one of the two first feature maps at the higher scale. For upsampling operation, Convolution operation, for Convolution operation, For splicing operations;

[0024] The first module performs the following operation on the first feature map at the lowest scale:

[0025] ;

[0026] in, The first feature map is at the lowest scale. For average pooling operation, This is a max pooling operation;

[0027] According to the first module, the and stated Perform the following operations:

[0028] ;

[0029] in, The first feature map at the lowest scale corresponds to the second feature map at the lowest scale. To enhance spatial attention operations, Enhanced channel attention operation, This is a pixel-by-pixel addition operation for features.

[0030] In some embodiments, the second module includes a global submodule and a local submodule;

[0031] The process of the second module outputting the third feature map includes:

[0032] The input feature map is input to the global submodule. Convolutional layer, then The output feature maps of the convolutional layers are respectively input into three... In the deep convolutional layer, three more The output feature maps of the deep convolutional layers are reshaped into , , Then, perform the following operations to obtain the output feature map of the global network:

[0033] ;

[0034] in, This is the output feature map of the global submodule. For normalized activation functions, These are weight values; Let be the input feature map, and let be the first feature map of any scale;

[0035] The input feature map is input into the local submodule, and the following operations are performed to obtain the output feature map of the local submodule:

[0036] ;

[0037] in, The output feature map of the local submodule. for Convolution operation, Shuffling operations for the channel;

[0038] The third feature map is obtained by performing the following operations:

[0039] ;

[0040] in, This is the third feature map.

[0041] In some embodiments, after obtaining the third feature map, the method includes:

[0042] The enhanced third feature map is obtained by performing the following operation on the third feature map:

[0043] ;

[0044] in, For N1 expansion rate The dilated convolution operation, For N2 expansion rate The dilated convolution operation, This is a pixel-wise feature multiplication operation;

[0045] The first fused feature map is obtained by fusing the multi-scale third feature map, including:

[0046] The enhanced third feature map from multiple scales is fused to obtain the first fused feature map.

[0047] In some embodiments, before converting the initial medical image into a first grayscale image, the method further includes:

[0048] The initial medical image is denoised.

[0049] In some embodiments, the initial medical image is a CT image.

[0050] To achieve the above objectives, a second aspect of the present invention provides an enhancement system for pathological examination medical images, the system comprising:

[0051] An image acquisition module is used to extract frames from medical images to obtain an initial medical image, and convert the initial medical image into a first grayscale image;

[0052] The feature extraction module is used to extract a multi-scale first feature map from the first grayscale image;

[0053] An edge texture detail enhancement module is used to input the lowest-scale first feature map and the higher-scale first feature map in the multi-scale first feature map into the first module, so as to enhance the edge texture features in the lowest-scale first feature map according to the first module and based on the higher-scale first feature map, to obtain the lowest-scale second feature map corresponding to the lowest-scale first feature map.

[0054] A global and local detail enhancement module is used to input the multi-scale first feature map into the second module, so as to output a multi-scale third feature map corresponding to the multi-scale first feature map according to the second module's fusion of convolution and attention mechanisms.

[0055] The feature fusion module is used to fuse the multi-scale third feature map to obtain a first fused feature map, and fuse the first fused feature map with the lowest-scale second feature map and the feature map of the first grayscale image to obtain a second fused feature map, and reconstruct the second fused feature map to obtain a second grayscale image.

[0056] An image conversion module is used to convert the second grayscale image into a final medical image.

[0057] To achieve the above objectives, a third aspect of the present invention provides an electronic device, comprising: at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory storing instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to perform the above-described method for enhancing medical images for pathological examination.

[0058] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the above-described method for enhancing medical images for pathological examination.

[0059] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a schematic diagram of a method for enhancing medical images for pathological examination provided in one embodiment of this application;

[0062] Figure 2(a) is a schematic diagram of the overall network provided in one embodiment of this application;

[0063] Figure 2(b) is a schematic diagram of the first module provided in one embodiment of this application;

[0064] Figure 2(c) is a schematic diagram of the second module provided in one embodiment of this application;

[0065] Figure 3 This is a schematic diagram of an enhancement system for pathological examination medical images provided in one embodiment of this application;

[0066] Figure 4 This is a schematic diagram of an electronic device provided in one embodiment of this application. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0068] CT imaging technology uses X-rays to penetrate the human body and obtains tomographic images through computer processing, providing high-resolution anatomical information. CT images have unique advantages in displaying bones and certain tissue structures. In clinical practice, doctors often need to identify tiny lesions and structural details from CT images; however, due to problems such as noise, artifacts, and insufficient contrast, these details are often difficult to identify.

[0069] Therefore, in order to improve the structural details of CT images, neural networks are currently widely used to enhance CT images. A common method is to use CNN (Convolutional Neural Network) enhancement, but it still has the following drawbacks:

[0070] The shallow features extracted by CNNs contain rich edge and non-edge details. However, CNNs rely too much on learning deep features, which leads to insufficient extraction of these details and poor CT image enhancement. Moreover, the limitations of convolutional operations in terms of local receptive fields may pose challenges in capturing global features, resulting in poor CT image enhancement.

[0071] like Figure 1 As shown in Figure 2, in order to solve the above-mentioned technical problems, embodiments are provided, including:

[0072] A method for enhancing medical images used in pathological examination, comprising the following steps S110 to S160:

[0073] Step S110: Extract frames from the medical image to obtain an initial medical image, and convert the initial medical image into a first grayscale image;

[0074] Step S120: Extract the first feature map of multiple scales from the first grayscale image;

[0075] Step S130: Input the lowest-scale first feature map and the higher-scale first feature map in the multi-scale first feature map into the first module, so as to enhance the edge texture features in the lowest-scale first feature map according to the first module and based on the higher-scale first feature map, to obtain the lowest-scale second feature map corresponding to the lowest-scale first feature map.

[0076] Step S140: Input the multi-scale first feature map into the second module, so as to output the multi-scale third feature map corresponding to the multi-scale first feature map according to the second module's fusion of convolution and attention mechanisms;

[0077] Step S150: Fuse the third feature map at multiple scales to obtain the first fused feature map, and fuse the first fused feature map with the second feature map at the lowest scale and the feature map of the first grayscale image to obtain the second fused feature map, and reconstruct the second grayscale image from the second fused feature map.

[0078] Step S160: Convert the second grayscale image into the final medical image.

[0079] In step S110, medical images refer to images acquired through various medical testing instruments. Frames are extracted from the medical images to obtain medical image frames (named initial medical images). The initial medical images are then converted to grayscale to obtain a first grayscale image. The purpose of grayscale conversion is to facilitate subsequent image enhancement processing.

[0080] In step S120, a multi-scale first feature map is extracted from the first grayscale image. This can be achieved through methods such as downsampling, multiple cascaded Transformer models, or multiple cascaded CNNs (e.g., feature pyramid structures). The shallow features (small-scale, high-resolution feature maps) in the multi-scale first feature map contain more details, including rich edge and texture details as well as non-edge and texture details. The deep features (large-scale, smaller-resolution feature maps) have a receptive field covering a larger area of ​​the image, enabling them to capture global semantic information.

[0081] In step S130, because the shallow features in the multi-scale first feature map contain more details, this step inputs the lowest-scale first feature map (i.e., shallow features) and the higher-scale first feature map (i.e., deep features) into the first module. Based on the first module, and utilizing information from the deep features, the learning of edges and textures, as well as non-edges and textures, in the shallow features is enhanced, resulting in the lowest-scale second feature map corresponding to the lowest-scale first feature map (i.e., the enhanced feature map corresponding to the shallow features). This approach has the advantage of strengthening the edges of the feature maps, thereby improving the image enhancement effect.

[0082] In step S140, the second module includes convolution and attention mechanisms. This is because convolution has limitations in terms of local receptive fields, and effectively capturing global features may be challenging. Therefore, attention mechanisms are added. By fusing convolution and attention mechanisms, global and local features can be learned effectively, ultimately improving the enhancement effect.

[0083] In step S150, by fusing the lowest-scale second feature map obtained in step S130, the multi-scale third feature map obtained in step S140, and the feature map of the first grayscale image extracted in step S110 (including, depending on the situation, including but not limited to, splicing, pixel-wise multiplication, and addition), the fused second feature map can have richer edge texture features and can effectively learn global and local features, thereby improving the feature enhancement effect.

[0084] In step S160, the second fused feature map is finally reconstructed into a second grayscale image, which is then converted into the final medical image.

[0085] In this method, the initial medical image is converted into a first grayscale image to facilitate subsequent image feature transformation. Then, multi-scale first feature maps are extracted from the first grayscale image. Next, a higher-scale first feature map and a lower-scale first feature map are selected. The higher-scale first feature map is used to enhance the edge, texture, and other features in the lower-scale first feature map, resulting in a lower-scale second feature map corresponding to the enhanced lower-scale first feature map. Then, through a second module that integrates convolution operations and attention mechanisms, a multi-scale third feature map with global and partial feature enhancements is output. Finally, the above feature maps are fused into a second fused feature map, and the second grayscale image is reconstructed, ultimately converting it into an enhanced medical image.

[0086] To fully extract the rich edge and non-edge texture details from the lowest-scale first feature map, this method first inputs the lowest-scale and higher-scale first feature maps into the first module. The first module then utilizes the edge and non-edge texture details from the higher-scale first feature map to facilitate the learning of edge and non-edge details in the shallow features. Next, by fusing convolution and attention mechanisms, the global and local features of the lesion structure are effectively learned. Finally, the images are fused, which overall improves the enhancement effect on medical images.

[0087] Further, as shown in Figure 2(a), the extraction of the first feature map from the first grayscale image in step S120 includes the following steps S210 and S220:

[0088] Step S210: Set up multiple Transformer blocks and cascade them; wherein the multiple Transformer blocks include at least four.

[0089] Step S220: Input the first grayscale image into multiple Transformer blocks to obtain the multi-scale first feature map output by multiple Transformer blocks.

[0090] As shown in Figure 2(a), the setup and cascading process of the Transformer block is as follows:

[0091] First, the first grayscale image is input into the first Transformer block, which processes the input image and outputs a feature map at the first scale.

[0092] Then, the feature map at the first scale is input into the second Transformer block for further processing, and the feature map at the second scale is output.

[0093] This process continues until the last Transformer block, which outputs the final multi-scale feature map.

[0094] In this way, feature maps at different scales can be extracted progressively, ensuring that important information in the image is fully captured. The structure of the Transformer block is common knowledge and will not be elaborated here.

[0095] Because CNNs have limitations in terms of local receptive field, this method extracts multi-scale first feature maps through multiple cascaded Transformer blocks, which can learn global features and make up for the shortcomings of CNNs in capturing global detailed features, ultimately improving the image enhancement effect.

[0096] Further, step S130, based on the first module and enhancing the edge texture features in the lowest-scale first feature map according to the higher-scale first feature map, to obtain the lowest-scale second feature map corresponding to the lowest-scale first feature map, includes:

[0097] Step S310: Select two high-scale first feature maps and the lowest-scale first feature map according to the first module in descending order of scale.

[0098] Step S320: Perform the following operations on the two high-scale first feature maps according to the first module:

[0099] ;

[0100] in, The first feature map at the higher scale is one of the two higher-scale first feature maps. The first feature map at the lower scale is one of the two first feature maps at the higher scale. For upsampling operation, Convolution operation, for Convolution operation, For splicing operations;

[0101] Step S330: Perform the following operations on the first feature map at the lowest scale according to the first module:

[0102] ;

[0103] in, The first feature map is at the lowest scale. For average pooling operation, This is a max pooling operation;

[0104] Step S340, for and Perform the following operations:

[0105] ;

[0106] in, The first feature map at the lowest scale corresponds to the second feature map at the lowest scale. To enhance spatial attention operations, Enhanced channel attention operation, This is a pixel-by-pixel addition operation for features.

[0107] As shown in Figure 2(b), in step S310, two high-scale first feature maps and one low-scale first feature map are selected in descending order of scale. For example, if there are four Transformer blocks, the feature maps output sequentially are... The feature map with the lowest scale (highest resolution) is ,in addition, The scale gradually increases. Here you can select... As the lowest-scale first feature map, select As the first feature map of two high-scale features.

[0108] In step S320, based on the above example, first... Upsampling, then with After convolution, the data is concatenated and then subjected to 1x1 convolution to maintain consistent channel dimensions.

[0109] In step S330, based on the above example, for After convolution, the inputs are fed into average pooling layers and max pooling layers, respectively, and then average and max pooling are performed separately before the results are output. Max pooling layers capture the most important features in a feature map, such as edges and textures, by using the maximum value. Average pooling layers smooth features by calculating the average value, thereby reducing the impact of noise.

[0110] In step S340, first the... and step 330 spliced ​​together Then respectively Spatial attention enhancement and channel attention enhancement yielded the following results: and GC attention focuses on the positional relationships between pixels, enhancing information such as feature map edges and textures, and can suppress noise; CA attention extracts channel information to focus on the importance of different channels, improving network robustness, and then... and By concatenating and convolving the channels, we obtain... Finally with Residual connections are used to alleviate the gradient vanishing problem, resulting in enhanced performance. That is, to obtain .

[0111] This method performs convolutional aggregation of shallow features and deep feature maps, which can improve the detail of lesion edges in medical images, thereby enhancing the accuracy of subsequent segmentation of lesion details and edges in medical images (subsequent processes after image enhancement).

[0112] Furthermore, as shown in Figure 2(c), the second module in step S140 includes a global submodule and a local submodule;

[0113] Furthermore, the process of the second module outputting the third feature map includes:

[0114] Step S410: Input the input feature map into the global submodule. Convolutional layer, then The output feature maps of the convolutional layers are respectively input into three... In the deep convolutional layer, three more The output feature maps of the deep convolutional layers are reshaped into , , Then perform the following operations to obtain the output feature map of the global network:

[0115] ;

[0116] in, This is the output feature map of the global submodule. For normalized activation functions, The weight values ​​are used to adjust the values ​​before applying the softmax operation. and The magnitude of matrix multiplication between them; Let be the input feature map, and let be the first feature map of any scale;

[0117] Step S420: Input the input feature map into the local submodule and perform the following operations to obtain the output feature map of the local submodule:

[0118] ;

[0119] in, The output feature map of the local submodule. for Convolution operation, Shuffling operations for the channel;

[0120] Step S430: Perform the following operations to obtain the third feature map:

[0121] ;

[0122] in, This is the third feature map.

[0123] This method introduces two branches: In the global branch, an attention mechanism is introduced to enhance long-distance information interaction. First, 1×1 convolution and 3×3 depthwise convolution operations are used to facilitate the transformation of the input feature map, thereby generating three different tensors, which are then reshaped separately. , , Then, attention maps are computed using Softmax normalization. In the local branch, in order to better interact and integrate features, a 1×1 convolution is first used to adjust the channel dimension. Then, the channel shuffling operation divides the input feature map into different groups along the channel axis. Depthwise separable convolutions are deployed in these groups to achieve channel shuffling, which is used to enhance the limitations of the convolution kernel in long-distance information interaction, enhance the global and local disease region features, and improve the enhancement effect.

[0124] Furthermore, after obtaining the third feature map, the method includes the following step S710:

[0125] Step S710: Perform the following operations on the third feature map to obtain the enhanced third feature map. :

[0126] ;

[0127] in, For N1 expansion rate The dilated convolution operation, For N2 expansion rate The dilated convolution operation, This is a pixel-wise feature multiplication operation;

[0128] The first fused feature map is obtained by fusing the third feature map from multiple scales, including:

[0129] The first fused feature map is obtained by fusing the enhanced third feature map from multiple scales.

[0130] In this method, since the aforementioned method is based on multi-scale feature map extraction using Transformer blocks, which contain a feedforward network (FFN), and the feedforward network is limited to single-scale feature aggregation, this embodiment adds two network branches. Both branches use 1×1 convolutions to adjust the channel dimensions. In one branch, a 3×3×3 depthwise convolution is used for feature extraction, and non-linearity is introduced through the ReLU activation function. In the other branch, two 3×3 dilated convolutional layers with dilation rates of N1 (which can be 2) and N2 (which can be 3) are used to improve the network's ability to extract a wider range of features. Then, element-wise multiplication operations are used to enhance the non-linear transformation of features. This method can enhance multi-scale information aggregation by extracting features across different scales, thereby solving the shortcomings of multi-scale feature map extraction using Transformer blocks and improving image enhancement results.

[0131] Furthermore, before converting the initial medical image into a first grayscale image, the method also includes:

[0132] Denoise the initial medical images.

[0133] Before being converted into a first-level grayscale image, the initial medical image undergoes denoising processing. Denoising reduces noise in the image, improves image quality, and makes subsequent feature extraction and enhancement processes more accurate and effective. Denoising effectively reduces the impact of artifacts and noise on image quality, thereby improving the clarity and recognizability of the final medical image.

[0134] Noise removal can be achieved using various methods. For example, non-local means (NLM) filtering can be used, which removes noise by calculating a weighted average of similar blocks in an image. Alternatively, wavelet transform denoising can be used, which filters out high-frequency noise in the wavelet domain by performing wavelet decomposition on the image. Deep learning-based denoising methods can also be employed, such as training a convolutional neural network (CNN) on a large number of noisy and clean images to enable the network to effectively remove noise.

[0135] This application effectively improves image quality by performing denoising processing on the initial medical image before converting it into a first grayscale image. This makes subsequent feature extraction and enhancement processing more accurate, resulting in a clearer and more identifiable medical image. Compared with existing technologies, this application adds a denoising step in the image preprocessing stage, further improving the enhancement effect of CT images.

[0136] Furthermore, the initial medical images are CT images. CT images provide high-resolution anatomical information, and by enhancing the image details in CT images, the accuracy of pathological examinations can be improved.

[0137] The technical solution of this application is achieved through the following steps:

[0138] (1) Extract frames from the CT images to obtain the initial medical image, and convert the initial medical image into a first grayscale image.

[0139] (2) Extract the first feature map of multiple scales from the first grayscale image.

[0140] (3) Input the lowest-scale first feature map and the higher-scale first feature map from the multi-scale first feature map into the first module to enhance the edge features in the lowest-scale first feature map and obtain the lowest-scale second feature map.

[0141] (4) Input the first feature map of multiple scales into the second module, fuse convolution and attention mechanisms, and output the third feature map of multiple scales.

[0142] (5) The third feature map of multiple scales is fused to obtain the first fused feature map, and then fused with the second feature map of the lowest scale and the first grayscale image to obtain the second fused feature map, and the second grayscale image is reconstructed.

[0143] (6) Convert the second grayscale image into the final medical image.

[0144] like Figure 3 In some embodiments of this application, an enhancement system for pathological examination medical images is also provided, the system comprising:

[0145] Image acquisition module 1001 is used to extract frames from medical images to obtain an initial medical image, and convert the initial medical image into a first grayscale image;

[0146] Feature extraction module 1002 is used to extract a first feature map of multiple scales from the first grayscale image;

[0147] The edge texture detail enhancement module 1003 is used to input the lowest-scale first feature map and the higher-scale first feature map in the multi-scale first feature map into the first module, so as to enhance the edge texture features in the lowest-scale first feature map according to the first module and based on the higher-scale first feature map, to obtain the lowest-scale second feature map corresponding to the lowest-scale first feature map.

[0148] The global and local detail enhancement module 1004 is used to input the multi-scale first feature map into the second module, so as to output the multi-scale third feature map corresponding to the multi-scale first feature map according to the fusion of convolution and attention mechanisms in the second module;

[0149] The feature fusion module 1005 is used to fuse the third feature map at multiple scales to obtain the first fused feature map, and to fuse the first fused feature map with the second feature map at the lowest scale and the feature map of the first grayscale image to obtain the second fused feature map, and to reconstruct the second grayscale image from the second fused feature map.

[0150] The image conversion module 1006 is used to convert a second grayscale image into a final medical image.

[0151] It should be noted that the enhancement system for pathological examination medical images provided in this embodiment is based on the same inventive concept as the enhancement method for pathological examination medical images described above. Therefore, the content related to the enhancement method for pathological examination medical images described above also applies to the content of the enhancement system for pathological examination medical images. Therefore, it will not be repeated here.

[0152] like Figure 4 This application also provides an electronic device, which includes:

[0153] At least one memory;

[0154] At least one processor;

[0155] At least one program;

[0156] The program is stored in memory, and the processor executes at least one program to implement the above-described method for enhancing medical images for pathological examination.

[0157] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0158] The electronic devices according to embodiments of this application will now be described in detail.

[0159] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0160] The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to execute the enhancement method for pathological examination medical images according to the embodiments of this invention.

[0161] To fully extract the rich edge and non-edge texture details from the lowest-scale first feature map, this method first inputs the lowest-scale and higher-scale first feature maps into the first module. The first module then utilizes the edge and non-edge texture details from the higher-scale first feature map to facilitate the learning of edge and non-edge details in the shallow features. Next, by fusing convolution and attention mechanisms, the global and local features of the lesion structure are effectively learned. Finally, the images are fused, which overall improves the enhancement effect on medical images.

[0162] The input / output interface 1800 is used to implement information input and output.

[0163] The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0164] Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900);

[0165] The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0166] This invention also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the above-described method for enhancing medical images for pathological examination.

[0167] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0168] The embodiments described in this invention are intended to more clearly illustrate the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0169] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present invention, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0170] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0171] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0172] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0173] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0174] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0175] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0176] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0177] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0178] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

Claims

1. A method for enhancing medical images used in pathological examination, characterized in that, The method includes: Frames are extracted from medical images to obtain an initial medical image, and the initial medical image is converted into a first grayscale image; Extracting a multi-scale first feature map from the first grayscale image; the extraction of the multi-scale first feature map from the first grayscale image includes: setting multiple Transformer blocks and cascading them; wherein the multiple Transformer blocks include at least four; inputting the first grayscale image into the multiple Transformer blocks to obtain the multi-scale first feature map output by the multiple Transformer blocks; The lowest-scale first feature map and the higher-scale first feature map in the multi-scale first feature map are input into the first module to enhance the edge texture features of the lowest-scale first feature map based on the higher-scale first feature map, thereby obtaining the lowest-scale second feature map corresponding to the lowest-scale first feature map; the step of enhancing the edge texture features of the lowest-scale first feature map based on the higher-scale first feature map to obtain the lowest-scale second feature map corresponding to the lowest-scale first feature map includes: According to the first module, two high-scale first feature maps and one low-scale first feature map are selected in descending order of scale. The first module performs the following operations on the two high-scale first feature maps: ； in, The first feature map at the higher scale is one of the two higher-scale first feature maps. The first feature map at the lower scale is one of the two first feature maps at the higher scale. For upsampling operation, Convolution operation, for Convolution operation, For splicing operations; The first module performs the following operation on the first feature map at the lowest scale: ； in, The first feature map is at the lowest scale. For average pooling operation, This is a max pooling operation; According to the first module, the and stated Perform the following operations: ； in, The first feature map at the lowest scale corresponds to the second feature map at the lowest scale. To enhance spatial attention operations, Enhanced channel attention operation, This is a pixel-by-pixel addition operation for features; The first feature map of the multi-scale is input into the second module, so that the third feature map of the multi-scale corresponding to the first feature map of the multi-scale is output according to the second module's fusion of convolution and attention mechanisms. The first fused feature map is obtained by fusing the third feature map of the multi-scale, and the first fused feature map is fused with the second feature map of the lowest scale and the feature map of the first grayscale image to obtain the second fused feature map. The second fused feature map is then reconstructed to obtain the second grayscale image. The second grayscale image is converted into the final medical image.

2. The enhancement method for pathological examination medical images according to claim 1, characterized in that, The second module includes global submodules and local submodules; The process of the second module outputting the third feature map includes: The input feature map is input to the global submodule. Convolutional layer, then The output feature maps of the convolutional layers are respectively input into three... In the deep convolutional layer, three more The output feature maps of the deep convolutional layers are reshaped into , , Then, perform the following operations to obtain the output feature map of the global network: ； in, This is the output feature map of the global submodule. For normalized activation functions, These are weight values; Let be the input feature map, and let be the first feature map of any scale; The input feature map is input into the local submodule, and the following operations are performed to obtain the output feature map of the local submodule: ； in, The output feature map of the local submodule. for Convolution operation, Shuffling operations for the channel; The third feature map is obtained by performing the following operations: ； in, This is the third feature map.

3. The enhancement method for pathological examination medical images according to claim 2, characterized in that, After obtaining the third feature map, the method includes: The enhanced third feature map is obtained by performing the following operation on the third feature map: ； in, For N1 expansion rate The dilated convolution operation, For N2 expansion rate The dilated convolution operation, This is a pixel-wise feature multiplication operation; The first fused feature map is obtained by fusing the multi-scale third feature map, including: The enhanced third feature map from multiple scales is fused to obtain the first fused feature map.

4. The method for enhancing medical images used in pathological examination according to claim 1, characterized in that, Before converting the initial medical image into a first grayscale image, the method further includes: The initial medical image is denoised.

5. The method for enhancing medical images used in pathological examination according to claim 1, characterized in that, The initial medical image is a CT image.

6. An enhancement system for pathological examination medical images, characterized in that, The system includes: An image acquisition module is used to extract frames from medical images to obtain an initial medical image, and convert the initial medical image into a first grayscale image; The feature extraction module is used to extract a multi-scale first feature map from the first grayscale image; the extraction of the multi-scale first feature map from the first grayscale image includes: Configure multiple Transformer blocks and cascade them; wherein the multiple Transformer blocks include at least four; The first grayscale image is input into the plurality of Transformer blocks to obtain the multi-scale first feature map output by the plurality of Transformer blocks; An edge texture detail enhancement module is used to input the lowest-scale first feature map and the higher-scale first feature map from the multi-scale first feature map into a first module, so as to enhance the edge texture features of the lowest-scale first feature map according to the first module and based on the higher-scale first feature map, to obtain a second feature map of the lowest scale corresponding to the lowest-scale first feature map; the step of enhancing the edge texture features of the lowest-scale first feature map according to the first module and based on the higher-scale first feature map to obtain a second feature map of the lowest scale corresponding to the lowest-scale first feature map includes: According to the first module, two high-scale first feature maps and one low-scale first feature map are selected in descending order of scale. The first module performs the following operations on the two high-scale first feature maps: ； in, The first feature map at the higher scale is one of the two higher-scale first feature maps. The first feature map at the lower scale is one of the two first feature maps at the higher scale. For upsampling operation, Convolution operation, for Convolution operation, For splicing operations; The first module performs the following operation on the first feature map at the lowest scale: ； in, The first feature map is at the lowest scale. For average pooling operation, This is a max pooling operation; According to the first module, the and stated Perform the following operations: ； in, The first feature map at the lowest scale corresponds to the second feature map at the lowest scale. To enhance spatial attention operations, Enhanced channel attention operation, This is a pixel-by-pixel addition operation for features; A global and local detail enhancement module is used to input the multi-scale first feature map into the second module, so as to output a multi-scale third feature map corresponding to the multi-scale first feature map according to the second module's fusion of convolution and attention mechanisms. The feature fusion module is used to fuse the multi-scale third feature map to obtain a first fused feature map, and fuse the first fused feature map with the lowest-scale second feature map and the feature map of the first grayscale image to obtain a second fused feature map, and reconstruct the second fused feature map to obtain a second grayscale image. An image conversion module is used to convert the second grayscale image into a final medical image.

7. An electronic device, characterized in that, include: At least one control processor and a memory for communicatively connecting to the at least one control processor; The memory stores instructions executable by the at least one control processor, which, when executed, enables the at least one control processor to perform the enhancement method for pathological examination medical images as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the enhancement method for pathological examination medical images as described in any one of claims 1 to 5.

Images