Medical image recognition method and system based on deep learning, electronic device and medium

Abstract

The application relates to the technical field of medical imaging, and discloses a medical image recognition method and system based on deep learning, an electronic device and a medium, which perform self-adaptive normalization on an initial medical image, and perform preprocessing on the initial medical image by using elastic deformation, random scaling and Gaussian noise to obtain a preprocessed medical image, construct a Swin-ResNeSt hybrid model based on the preprocessed medical image, add an adaptive multi-scale attention module to the Swin-ResNeSt hybrid model, design a double-task head including classification and positioning, adopt a hybrid loss function combined by a Focal Loss function and a GIoU Loss function, and output a lesion category and a bounding box; and the application improves recognition accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical imaging, in particular to a medical image recognition method and system based on deep learning, an electronic device and a medium. BACKGROUND

[0002] With the rapid development of medical imaging technology, clinical diagnosis puts forward higher requirements on the accuracy and efficiency of image analysis. The existing medical image recognition method based on deep learning mainly has the following problems: the traditional convolutional neural network performs unevenly when processing different sizes of lesions, small lesions are easily ignored, and the boundary recognition of large lesions is not accurate; the medical image data annotation cost is high, the sample size is limited, and the model generalization ability is insufficient. SUMMARY

[0003] The present application relates to the technical field of medical imaging, in particular to a medical image recognition method and system based on deep learning, an electronic device and a medium.

[0004] The first aspect of the present application provides a medical image recognition method based on deep learning, which comprises the following steps:

[0005] The initial medical image is subjected to split modality adaptive normalization, and is preprocessed by elastic deformation, random scaling and Gaussian noise to obtain a preprocessed medical image;

[0006] A Swin-ResNeSt hybrid model is constructed based on the preprocessed medical image, wherein the bottom layer of the Swin-ResNeSt hybrid model extracts local features through a ResNeSt network, the middle layer models global dependence through a Swin Transformer network, and contrastive learning pre-training is completed on unlabeled medical images based on a MoCo v3 framework;

[0007] An adaptive multi-scale attention module is added to the Swin-ResNeSt hybrid model, wherein the adaptive multi-scale attention module includes a scale perception weight calculation unit, a spatial adaptive adjustment unit and a scale fusion unit;

[0008] A dual-task head including classification and positioning is designed, a hybrid loss function combining a Focal Loss function and a GIoU Loss function is adopted, and a lesion category and a bounding box are output.

[0009] Optionally, in the first implementation manner of the first aspect of the present application, the split modality adaptive normalization of the initial medical image and the preprocessing by elastic deformation, random scaling and Gaussian noise to obtain the preprocessed medical image comprise:

[0010] The initial medical image is classified according to a modality type, and a modality-specific adaptive normalization operation is performed on different modalities to obtain a normalized image, wherein the modality type at least includes CT, MRI T1 weighted image and T2 weighted image;

[0011] A smooth displacement field is generated by Gaussian filtering, and the position of each pixel in the normalized image is adjusted according to the displacement field to perform elastic deformation processing;

[0012] A scaling factor is randomly selected from a scaling factor range of 0.8-1.2, and the elastic deformation image is randomly scaled according to the scaling factor, and the image is restored to the original size after scaling by padding;

[0013] A Gaussian noise matrix with the same size as the image is generated according to a preset noise intensity, and the Gaussian noise matrix and the image pixel value are added at the corresponding positions, and the Gaussian noise is added to the scaled image to obtain the preprocessed medical image.

[0014] Optionally, in the second implementation manner of the first aspect of the present application, the Swin-ResNeSt hybrid model is constructed based on the preprocessed medical image, comprising:

[0015] The first medical image is input into the bottom network, and the local texture, edge and other detail features of the image are extracted layer by layer through multi-layer convolution operation with grouping attention, and a multi-scale local feature map is output, wherein the ResNeSt network introduces a grouping attention mechanism in the residual block;

[0016] The middle network is connected at the output end of the bottom network, and the SwinTransformer network structure is used to divide the local feature map into a plurality of non-overlapping square windows; self-attention calculation is performed on the features in each window to obtain local correlation features in the window;

[0017] The attention calculation across windows is realized through the sliding window mechanism to capture the global dependency relationship between different windows, and the middle network outputs a feature map containing global context information;

[0018] A pre-training double-branch network is built based on the MoCo v3 framework, the unlabeled medical image is input into the double-branch network, two different views of the same image are generated through random data augmentation, and feature vectors are obtained through query branch and key branch processing respectively;

[0019] The InfoNCE loss of the feature vectors obtained by query branch and key branch processing is calculated, the query branch parameters are updated according to the preset training rounds, learning rate and batch size, and the contrastive learning pre-training is completed.

[0020] Optionally, in a third implementation form of the first aspect of the present application, the dual-branch network comprises a query branch and a key branch, wherein the query branch is a Swin-ResNeSt hybrid network, and the key branch adopts a network with the same structure as the query branch but independent parameters.

[0021] Optionally, in a fourth implementation form of the first aspect of the present application, the adaptive multi-scale attention module added in the Swin-ResNeSt hybrid model comprises:

[0022] The scale-aware weight calculation unit is connected to an output end of the middle-layer network of the Swin-ResNeSt hybrid model, and multi-scale feature maps output by the middle-layer network are collected;

[0023] For each scale of feature map, feature compression is performed through a 3*3 lightweight convolution layer, and then the compressed feature is input into a Sigmoid activation function to obtain an initial weight of each scale of feature map;

[0024] The spatial adaptive adjustment unit is started to calculate a pixel-level spatial distribution entropy for each scale of feature map, and the attention weight of the corresponding position is dynamically adjusted according to the entropy value;

[0025] The multi-scale feature map after spatial adaptive adjustment is fused through the scale fusion unit, each scale of feature map is multiplied by the corresponding initial weight and spatial adjustment weight to obtain a weighted feature map;

[0026] All weighted feature maps are added according to pixel positions to generate a feature representation fused with multi-scale information and having spatial adaptive attention.

[0027] Optionally, in a fifth implementation form of the first aspect of the present application, the design comprises a dual-task head for classification and positioning, a hybrid loss function combining a Focal Loss function and a GIoU Loss function is adopted, and a lesion category and a bounding box are output, comprising:

[0028] In the classification task head, the feature representation is input into a two-layer fully connected network, a Sigmoid activation function is applied to the feature output by the second layer to obtain a probability value of each category;

[0029] In the positioning task head, a full convolution network structure is adopted to perform multi-layer convolution operation on the feature representation, and a feature map with the same size as the image is output, wherein each pixel position corresponds to a group of coordinate parameters for representing a possible lesion bounding box at the current position;

[0030] The gradient descent algorithm is used to update the parameters of the Swin-ResNeSt hybrid model, and after the training is completed, the medical image to be identified is input into the Swin-ResNeSt hybrid model, and the lesion category is output through the classification task head, and the lesion boundary box is output through the positioning task head.

[0031] Optionally, in a sixth implementation manner of the first aspect of the present application, the first full connection network outputs intermediate features of a preset dimension, and the second full connection network outputs features corresponding to the number of lesion categories.

[0032] The second aspect of the present application provides a medical image recognition system based on deep learning, which comprises:

[0033] A preprocessing module is configured to perform self-adaptive normalization on the initial medical image, and perform preprocessing on the medical image by using elastic deformation, random scaling and Gaussian noise to obtain a preprocessed medical image.

[0034] A model construction module is configured to construct a Swin-ResNeSt hybrid model based on the preprocessed medical image, wherein the Swin-ResNeSt hybrid model extracts local features through a ResNeSt network at the bottom layer, models global dependence through a Swin Transformer network at the middle layer, and completes contrastive learning pre-training on unlabeled medical images based on a MoCo v3 framework.

[0035] An adding module is configured to add an adaptive multi-scale attention module to the Swin-ResNeSt hybrid model, wherein the adaptive multi-scale attention module comprises a scale perception weight calculation unit, a spatial adaptive adjustment unit and a scale fusion unit.

[0036] An output module is configured to design a double-task head including classification and positioning, adopt a hybrid loss function combining a Focal Loss function and a GIoU Loss function, and output a lesion category and a boundary box.

[0037] The third aspect of the present application provides an electronic device, which comprises a memory and at least one processor, and the memory stores instructions; the at least one processor invokes the instructions in the memory to enable the deep learning-based medical image recognition device to perform the steps of the deep learning-based medical image recognition method according to any one of the above.

[0038] The fourth aspect of the present application provides a computer readable storage medium, which stores instructions, and the instructions are executed by a processor to implement the steps of the deep learning-based medical image recognition method according to any one of the above.

[0039] The technical scheme provided by the application comprises the following steps: performing self-adaptive normalization on an initial medical image in a mode, and performing preprocessing on the medical image by using elastic deformation, random scaling and Gaussian noise to obtain a preprocessed medical image; constructing a Swin-ResNeSt hybrid model based on the preprocessed medical image; adding an adaptive multi-scale attention module in the Swin-ResNeSt hybrid model; designing a double-task head comprising classification and positioning; adopting a hybrid loss function combining a Focal Loss function and a GIoU Loss function; and outputting a lesion category and a bounding box; the adaptive multi-scale attention mechanism can dynamically adjust the weights of features of different scales according to image content, effectively solves the problem of imbalance in multi-size lesion recognition, improves the recognition accuracy, and meets the recognition requirements of a complex image environment in a clinic. BRIEF DESCRIPTION OF DRAWINGS

[0040] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings provided herein are for illustrative purposes only and are not considered a limitation on the application.

[0041] Figure 1 A flowchart of a medical image recognition method based on deep learning provided for an embodiment of the application is shown in FIG. 1.

[0042] Figure 2 A structural schematic diagram of a medical image recognition system based on deep learning provided for an embodiment of the application is shown in FIG. 2.

[0043] Figure 3 A structural schematic diagram of a medical image recognition device based on deep learning provided for an embodiment of the application is shown in FIG. 3. DETAILED DESCRIPTION

[0044] The terms "first", "second", "third", "fourth" and the like in the description, claims, and drawings of the application, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of these terms herein is to be construed as interchangeable in order to distinguish between the similar objects. It is also to be understood that the terms "comprising", "having", "including" and "containing" are to be construed as open-ended terms (i.e., meaning "including, but not limited to") unless otherwise noted to have a closed meaning (i.e., meaning "consisting of"). Thus, use of such terms is not intended to preclude the addition of a non-exhaustive list of elements, such as steps or components, to a process, method, device, product or apparatus.

[0045] For the purpose of facilitating understanding, the specific flow of the embodiments of the application is described below. Please refer to Figure 1The flowchart of the medical image recognition method based on deep learning provided by the embodiment of the application specifically comprises the following steps:

[0046] In step 101, the initial medical image is subjected to self-adaptive normalization by mode, and is preprocessed by elastic deformation, random scaling and Gaussian noise to obtain a preprocessed medical image.

[0047] In this embodiment, the initial medical image is accurately classified according to the mode type, and the CT image is clearly distinguished from the T1 weighted image and the T2 weighted image of MRI. This is because there are significant differences in physical characteristics and signal meanings between different modal images. The pixel value of the CT image is expressed in HU (Hounsfield Unit), which reflects the tissue density. The physiological significance interval of-1000~400HU needs to be intercepted, which covers the key areas such as the lung and soft tissue. Then, the interval value is converted to the standardized range of 0~1 through linear mapping to avoid interference from irrelevant density values. The T1 weighted image of MRI focuses on displaying anatomical structures, such as fat showing high signal, and the T2 weighted image is more likely to distinguish between edema and lesion tissues, such as liquid showing high signal. Therefore, the mean and standard deviation of all pixels under each mode are calculated respectively. Each pixel value is subtracted from the mean and then divided by the standard deviation to achieve the gray consistency of the images within the same mode. Finally, the normalized image is obtained, which lays a unified foundation for subsequent feature extraction.

[0048] Elastic deformation processing simulates the natural movement state of human organs such as the lung and the brain during the process of breathing and heartbeat, thereby enhancing the adaptability of the model to the position changes of the organs. Specifically, a continuous and smooth displacement field is generated based on preset Gaussian filtering parameters such as filter kernel size and standard deviation. The displacement field ensures that the pixel movement amplitude gradually changes with the spatial position to avoid abrupt jumps. Then, the new positions of the corresponding pixels in the normalized image are recalculated according to the offset of each coordinate point in the displacement field. For the pixels that exceed the original image range after displacement, the edge pixel value is filled. For the pixels in the overlapping area, the final value is determined by an interpolation method such as bilinear interpolation, so that the deformed image not only retains the relevance of the original anatomical structure, but also has the diversity of the movement state.

[0049] Random scaling processing aims to adapt to the resolution differences of images collected by different medical devices, such as different layer thicknesses and fields of view of different CT devices, thereby improving the cross-device robustness of the model. When operating, a scaling coefficient is randomly selected from the range of 0.8~1.2. This range not only covers the commonly seen resolution fluctuation range in clinical practice, but also avoids structural distortion caused by excessive scaling. After the image after elastic deformation is scaled by the selected coefficient, the edge filling method such as copying the edge pixel value of the image to the four directions is used to restore the image to the original size, so that the scaled image maintains the integrity of the key lesion structure while its spatial dimension is consistent with the original input.

[0050] The Gaussian noise is added to simulate the device noise that can exist in the clinical image, such as the quantum noise of low-dose CT and the electromagnetic interference noise of MRI, so as to improve the anti-noise ability of the enhancement model. First, a Gaussian noise matrix with the same size as the image is generated according to the preset noise intensity, such as the noise standard deviation, which is usually set to 0.01-0.03 to match the typical noise level of the clinical image. The pixel value of the matrix obeys the normal distribution with a mean of 0 and a preset standard deviation. Then, the noise matrix and the scaled image are added one by one according to the pixel position, so that the noise is uniformly distributed in the entire image. Finally, the preprocessed medical image is obtained. This processing not only preserves the original lesion characteristics of the image, but also artificially introduces controllable noise, forcing the model to learn more robust lesion recognition features and reducing the interference of noise on the recognition result.

[0051] In step 102, a Swin-ResNeSt hybrid model is constructed based on the preprocessed medical image. The bottom layer of the Swin-ResNeSt hybrid model extracts local features through the ResNeSt network, the middle layer models global dependence through the Swin Transformer network, and the contrastive learning pre-training is completed on the unlabeled medical image based on the MoCo v3 framework.

[0052] In this embodiment, the first medical image is input into the bottom network ResNeSt network of the Swin-ResNeSt hybrid model. The network innovatively introduces a grouping attention mechanism based on the traditional residual block. First, the feature channels output by the residual block are divided into several non-overlapping groups. The channel attention weight is calculated in each group through a fully connected layer and a softmax activation function. The weight value reflects the importance of the channel feature to lesion recognition. Then, a multi-layer convolution operation with grouping attention is performed. The size of each convolution kernel is 3x3, and the step increases with the depth of the network to realize down-sampling and extract local features at different levels in the image. The shallow network captures pixel-level texture such as gray level change of lesion edge, the middle network extracts local structure such as circular outline of small nodules, and the deep network aggregates more abstract detail features such as density difference between lesion and surrounding tissue. Finally, multi-scale local feature maps of 1 / 2, 1 / 4, 1 / 8, 1 / 16, etc. are output.

[0053] The middle layer network SwinTransformer network is connected at the output end of the bottom layer network. First, the multi-scale local feature map is divided into a plurality of non-overlapping square windows according to a preset window size, such as 7x7 pixels. The selection of the window size takes into account the calculation efficiency and the integrity of the local features to ensure that small lesions are completely contained in a single window. For the features in each window, a query Query, a key Key, and a value Value matrix are generated through linear mapping. The similarity between the query and the key is calculated to obtain an attention score. After softmax normalization, the value matrix is weighted and summed to complete the self-attention calculation in the window. This process can capture the spatial correlation between pixels within the window, such as the continuous texture of the lesion edge. Finally, the local correlation features of each window are obtained, enhancing the feature expression of small-range lesion structures.

[0054] Cross-window attention calculation is achieved through a sliding window mechanism. The divided windows are slid along the horizontal and vertical directions by a preset step, such as 1 / 2 of the window size, causing adjacent windows to have overlapping regions. For the features in the overlapping regions, the attention scores between different windows are recalculated, which is the correlation strength between the features of one window and the features of adjacent windows. Through this cross-window interaction, the spatial limitations of a single window are broken, and global dependency relationships between different windows are captured, such as the positional correlation between pulmonary nodules and blood vessels, and the spatial distribution of brain tumors and brain ventricles. After multiple layers of cross-window attention calculation, the middle layer network outputs a feature map that integrates local details and global context. This feature map not only retains the fine structure information of the lesion but also contains its spatial positioning information in the entire organ.

[0055] A pre-training dual-branch network is built based on the MoCo v3 framework. The query branch directly uses the Swin-ResNeSt hybrid network constructed above, and the key branch uses a network that is identical in structure but independent in parameters to the query branch. The key branch parameters do not participate in gradient update and are only associated with the query branch through the momentum mechanism. A large amount of unlabeled medical images, such as unlabeled CT and MRI sequences, are input into the dual-branch network. Different data augmentation strategies, such as different degrees of elastic deformation, scaling, and noise injection, are randomly applied to each image to generate two different views, which are input into the query branch and the key branch, respectively. The two branches output high-dimensional feature vectors through their respective feature extraction processes, including bottom layer local features and middle layer global features. The query branch features are used for subsequent loss calculation, and the key branch features serve as a comparison benchmark.

[0056] Computing the InfoNCE loss of the query branch and the key branch output feature vector: regarding the features of two views of the same image as a positive sample pair, comparing them with the view features of other images in the batch as negative sample pairs, calculating the similarity probability of the positive sample pair by the softmax function, and constructing a loss function with the goal of maximizing the positive sample probability and minimizing the negative sample probability; updating only the network parameters of the query branch according to the preset training parameters such as 100 training rounds, an initial learning rate of 5e-4, and a batch size of 64, and the key branch parameters slowly track the query branch parameters through a momentum coefficient of 0.999; after complete pre-training, the query branch, i.e., the Swin-ResNeSt hybrid network, has learned the general feature rules of medical images from unannotated data, such as the anatomical structure distribution of organs and the common features of lesions.

[0057] Step 103, adding an adaptive multi-scale attention module to the Swin-ResNeSt hybrid model, wherein the adaptive multi-scale attention module includes a scale-aware weight calculation unit, a spatial adaptive adjustment unit, and a scale fusion unit.

[0058] In this embodiment, the scale-aware weight calculation unit is connected to the output end of the middle layer network of the Swin-ResNeSt hybrid model. The primary task of this unit is to collect the multi-scale feature maps output by the middle layer network. These feature maps are derived from the global dependency modeling of the bottom layer local features by the middle layer SwinTransformer network and cover four typical resolutions of 1 / 2, 1 / 4, 1 / 8, and 1 / 16. Among them, the high-resolution features of 1 / 2 and 1 / 4 retain more details such as lesion edges and textures, while the low-resolution features of 1 / 8 and 1 / 16 contain global association information between the lesion and the surrounding tissue. Figure 1 Figure 1 The unit stores the multi-scale feature maps in order of resolution through a preset feature receiving interface, providing structured input for subsequent weight calculation for different scales and avoiding weight distribution bias caused by confusion of different scale features.

[0059] For each resolution feature map, a feature compression operation is first performed: a 3x3 lightweight convolution layer is used to perform convolution calculation on the feature map. The number of convolution kernels of this convolution layer is only 1 / 4 of the number of channels of the corresponding feature map, for example, if the original feature map contains 256 channels, the convolution layer is set to have 64 convolution kernels. This reduces the amount of calculation while capturing the association information of the local neighborhood of the feature map through a 3x3 receptive field, achieving dimension reduction and redundancy filtering of the feature channels. Subsequently, the compressed single-channel feature map is input into the Sigmoid activation function, which maps the feature values to the interval of 0-1. The mapping result is the initial weight of the corresponding scale feature map. The closer the weight value is to 1, the higher the contribution of the scale feature map to lesion recognition, and the closer it is to 0, the lower the contribution. This preliminarily realizes the importance differentiation of different scale features. ​

[0060] After starting the spatial self-adaptive adjustment unit, the unit first calculates the pixel-level spatial distribution entropy of each scale feature map: for the 8-neighborhood region of each pixel in the feature map, the distribution frequency of the feature values in the region is counted, and then the entropy value of the pixel position is calculated through the information entropy formula such as Shannon entropy. The higher the entropy value, the more intense the feature change around the pixel, and the more information-intensive areas such as lesion edges and lesion cores are likely to be; the lower the entropy value, the more relatively flat the region, and the more normal tissues or background areas are; based on the entropy value result, the unit dynamically adjusts the attention weight of each pixel position: the entropy value is divided into high, medium and low intervals according to a preset threshold, the weight of the high entropy interval is increased to 1.5-2 times the original weight, the weight of the medium entropy interval remains the original weight, and the weight of the low entropy interval is reduced to 0.3-0.5 times the original weight, so that the attention is more focused on the key spatial areas related to the lesion.

[0061] The scale fusion unit first retrieves the initial weight corresponding to each scale feature map from the scale perception weight calculation unit and the spatial adjustment weight from the spatial self-adaptive adjustment unit, and then performs double weighting on each scale feature map according to the calculation logic of feature map x initial weight x spatial adjustment weight: the initial weight selects high-contribution features from the scale dimension, and the spatial adjustment weight enhances key region features from the pixel dimension, and the combination of the two realizes double suppression of background areas with low-contribution scales; for example, in the high-initial-weight 1 / 2 resolution feature map, the high-entropy pixels of the lesion edge will be given a higher comprehensive weight, while the low-entropy pixels in the low-initial-weight 1 / 16 resolution feature map will be given a very low comprehensive weight, and finally the weighted feature map of each scale is obtained, ensuring that the effective features are highlighted and retained.

[0062] Perform pixel-level addition fusion on all weighted multi-scale feature maps: the weighted feature maps of 1 / 2, 1 / 4, 1 / 8 and 1 / 16 resolution are uniformly adjusted to 1 / 4 resolution by upsampling such as bilinear interpolation, taking into account detail retention and computational efficiency, and then added one by one according to the corresponding pixel positions, and the final value of each pixel is the sum of the values of the weighted feature maps of each scale at that position; this fusion method can aggregate the effective information of different scales, high-resolution detail features and low-resolution global features into the same feature map, while avoiding the interference of background and redundant information on the fusion result due to the previous double weighting screening; the final generated feature representation contains multi-scale lesion information and has the spatial self-adaptive attention focusing property.

[0063] Step 104, design a dual-task head including classification and positioning, adopt a hybrid loss function combining Focal Loss function and GIoU Loss function, and output lesion categories and bounding boxes.

[0064] In this embodiment, the feature representation is input into a two-layer fully connected network in the classification task head, the first layer of the fully connected network outputs intermediate features of a preset dimension, and the second layer of the fully connected network outputs features corresponding to the number of lesion categories. The Sigmoid activation function is applied to the features output by the second layer to obtain probability values for each category. In the positioning task head, a full convolutional network structure is used to perform multi-layer convolution operations on the feature representation, and a feature map with the same size as the image is output, wherein each pixel position corresponds to a group of coordinate parameters for representing a possible lesion bounding box at the current position. The gradient descent algorithm is used to update the parameters of the Swin-ResNeSt hybrid model, and after training, the medical image to be identified is input into the Swin-ResNeSt hybrid model, the lesion category is output through the classification task head, and the lesion bounding box is output through the positioning task head.

[0065] Referring to Figure 2 The medical image recognition system based on deep learning provided in the embodiment of the present application has the following advantages:

[0066] The preprocessing module is configured to perform modality-adaptive normalization on the initial medical image, and perform preprocessing on the initial medical image by using elastic deformation, random scaling and Gaussian noise to obtain a preprocessed medical image.

[0067] The model construction module is configured to construct a Swin-ResNeSt hybrid model based on the preprocessed medical image, wherein the Swin-ResNeSt hybrid model extracts local features through a ResNeSt network at a bottom layer, models global dependencies through a Swin Transformer network at a middle layer, and completes contrastive learning pre-training on unlabeled medical images based on a MoCo v3 framework.

[0068] The adding module is configured to add an adaptive multi-scale attention module to the Swin-ResNeSt hybrid model, wherein the adaptive multi-scale attention module includes a scale perception weight calculation unit, a spatial adaptive adjustment unit and a scale fusion unit.

[0069] The output module is configured to design a double-task head including classification and positioning, and use a hybrid loss function combining a Focal Loss function and a GIoU Loss function to output the lesion category and the bounding box.

[0070] Figure 3is a structural schematic view of a medical image recognition device based on deep learning provided by an embodiment of the present application. The medical image recognition device based on deep learning 300 can have great differences due to different configurations or performances, and can include one or more processors (central processing units, CPUs) 310 (for example, one or more processors) and a memory 320, one or more storage media 330 (for example, one or more mass storage devices) storing application programs 333 or data 332. The memory 320 and the storage media 330 can be temporary storage or persistent storage. The programs stored in the storage media 330 can include one or more modules (not shown in the figure), each of which can include a series of instruction operations in the medical image recognition device based on deep learning 300. Further, the processor 310 can be configured to communicate with the storage media 330 and execute a series of instruction operations in the storage media 330 on the medical image recognition device based on deep learning 300 to implement the method provided by the above embodiment.

[0071] The medical image recognition device based on deep learning 300 can also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 331, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that, Figure 3 The medical image recognition device based on deep learning shown in the structure does not constitute a limitation on the computer device provided by the present application, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0072] The present application also provides a computer readable storage medium, which can be a non-volatile computer readable storage medium, and can also be a volatile computer readable storage medium. The computer readable storage medium has instructions stored therein, and when the instructions are run on a computer, the computer executes the steps of the medical image recognition method based on deep learning provided by each of the above embodiments.

[0073] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device or apparatus, unit can refer to the corresponding process in the foregoing method embodiment, which will not be described here.

[0074] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0075] The basic principles, main features and advantages of the present application are shown and described above. Those skilled in the art should understand that the present application is not limited by the above embodiments, and the above embodiments and descriptions in the specification are only preferred examples of the present application and are not intended to limit the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A medical image recognition method based on deep learning, characterized by, The method comprises the following steps: The initial medical image is subjected to adaptive normalization by modality, and is preprocessed by elastic deformation, random scaling and Gaussian noise to obtain a preprocessed medical image; A Swin-ResNeSt hybrid model is constructed based on the preprocessed medical image, wherein the bottom layer of the Swin-ResNeSt hybrid model extracts local features through a ResNeSt network, the middle layer models global dependence through a Swin Transformer network, and contrast learning pretraining is completed on unlabeled medical images based on a MoCo v3 framework; An adaptive multi-scale attention module is added to the Swin-ResNeSt hybrid model, wherein the adaptive multi-scale attention module comprises a scale perception weight calculation unit, a spatial adaptive adjustment unit and a scale fusion unit; A double-task head comprising classification and positioning is designed, a hybrid loss function combining a Focal Loss function and a GIoU Loss function is adopted, and a lesion category and a bounding box are output. 2.The deep learning-based medical image recognition method of claim 1, wherein, The adaptive normalization by modality and the preprocessing of the initial medical image by elastic deformation, random scaling and Gaussian noise to obtain the preprocessed medical image comprise: The initial medical image is classified by modality type, and a modality-specific adaptive normalization operation is performed on different modalities to obtain a normalized image, wherein the modality types at least include CT, MRI T1 weighted image and T2 weighted image; A smooth displacement field is generated by Gaussian filtering, and the position of each pixel in the normalized image is adjusted according to the displacement field to perform elastic deformation processing; A scaling coefficient is randomly selected from a ratio range of 0.8-1.2, and the image after elastic deformation is randomly scaled by the scaling coefficient, and the image is restored to the original size after scaling by padding; A Gaussian noise matrix with the same size as the image is generated according to a preset noise intensity, the Gaussian noise matrix and the image pixel value are added at the corresponding positions, the Gaussian noise is added to the scaled image, and the preprocessed medical image is obtained. 3.The deep learning-based medical image recognition method of claim 1, wherein, The Swin-ResNeSt hybrid model is constructed based on the preprocessed medical image, comprising: The first medical image is input into the bottom layer network, and the local texture, edge detail features of the image are extracted layer by layer through multi-layer convolution operation with grouping attention, and a multi-scale local feature map is output, wherein the ResNeSt network introduces a grouping attention mechanism in the residual block; The output end of the bottom layer network is connected to the middle layer network, and the SwinTransformer network structure is used to divide the local feature map into a plurality of non-overlapping square windows; self-attention calculation is performed on the features in each window to obtain local correlation features in the window; Attention calculation across windows is realized through a sliding window mechanism to capture global dependence between different windows, and the middle layer network outputs a feature map containing global context information; The pre-training dual-branch network is built based on the MoCo v3 framework, the unlabeled medical image is input into the dual-branch network, two different views of the same image are generated through random data augmentation, and feature vectors are obtained through the query branch and the key branch; The InfoNCE loss of the feature vectors obtained by the query branch and the key branch is calculated, the query branch parameters are updated according to the preset training round, learning rate and batch size, and the contrast learning pre-training is completed. 4.The deep learning-based medical image recognition method of claim 3, wherein, The dual-branch network includes a query branch and a key branch, wherein the query branch is a Swin-ResNeSt hybrid network, and the key branch adopts a network with the same structure as the query branch but independent parameters. 5.The deep learning-based medical image recognition method of claim 1, wherein, The adaptive multi-scale attention module is added in the Swin-ResNeSt hybrid model, including: A scale perception weight calculation unit is connected to the output end of the middle layer network of the Swin-ResNeSt hybrid model, and the multi-scale feature maps output by the middle layer network are collected; For each scale feature map, first, a 3*3 lightweight convolution layer is used for feature compression, and then the compressed features are input into a Sigmoid activation function to obtain the initial weight of each scale feature map; A spatial adaptive adjustment unit is started, and the spatial distribution entropy of each scale feature map is calculated, and the attention weight of the corresponding position is dynamically adjusted according to the entropy value; Through the scale fusion unit, the multi-scale feature maps adjusted by the spatial adaptive adjustment unit are fused, and each scale feature map is multiplied by its corresponding initial weight and spatial adjustment weight to obtain a weighted feature map; All weighted feature maps are added according to the pixel position to generate a feature representation that fuses multi-scale information and has spatial adaptive attention. 6.The deep learning-based medical image recognition method of claim 5, wherein, The design includes a dual-task head for classification and positioning, adopts a hybrid loss function combining Focal Loss function and GIoU Loss function, and outputs lesion categories and bounding boxes, including: In the classification task head, the feature representation is input into a two-layer fully connected network, and a Sigmoid activation function is applied to the output features of the second layer to obtain probability values of each category; In the positioning task head, a full convolution network structure is used to perform multi-layer convolution operation on the feature representation, and a feature map with the same size as the image is output, wherein each pixel position corresponds to a group of coordinate parameters for representing the lesion bounding box existing at the current position; The gradient descent algorithm is used to update the parameters of the Swin-ResNeSt hybrid model, and after training, the medical image to be identified is input into the Swin-ResNeSt hybrid model, the lesion category is output through the classification task head, and the lesion bounding box is output through the positioning task head. 7.The deep learning-based medical image recognition method of claim 6, wherein, The first layer fully connected network outputs intermediate features of a preset dimension, and the second layer fully connected network outputs features corresponding to the number of lesion categories.

8. A medical image recognition system based on deep learning, characterized by, The system comprises: A preprocessing module is configured to perform modality-adaptive normalization on the initial medical image, and perform preprocessing such as elastic deformation, random scaling and Gaussian noise to obtain a preprocessed medical image; The model construction module is configured to construct a Swin-ResNeSt hybrid model based on the preprocessed medical image, wherein the Swin-ResNeSt hybrid model extracts local features through a ResNeSt network at a bottom layer, models global dependence through a Swin Transformer network at a middle layer, and completes contrast learning pre-training on unannotated medical images based on a MoCo v3 framework; The adding module is configured to add an adaptive multi-scale attention module to the Swin-ResNeSt hybrid model, wherein the adaptive multi-scale attention module comprises a scale perception weight calculation unit, a spatial adaptive adjustment unit, and a scale fusion unit; The output module is configured to design a double-task head including classification and positioning, adopt a hybrid loss function combining a Focal Loss function and a GIoU Loss function, and output a lesion category and a bounding box.

9. An electronic device, comprising: The electronic device includes a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to enable the deep learning-based medical image recognition device to perform the steps of the deep learning-based medical image recognition method according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon instructions, the instructions comprising, The instructions are executed by the processor to implement the steps of the deep learning-based medical image recognition method according to any one of claims 1-7.

Images