Medical image recognition method and system based on deep learning, electronic equipment and medium

Abstract

The invention relates to the technical field of medical images, and discloses a medical image recognition method and system based on deep learning, electronic equipment and a medium, and the method comprises the steps: carrying out the split-mode adaptive normalization of an initial medical image, carrying out the preprocessing through elastic deformation, random zooming and Gaussian noise, and obtaining a preprocessed medical image, constructing a Swinin-ResNeSt hybrid model based on the preprocessed medical image, adding an adaptive multi-scale attention module in the Swinin-ResNeSt hybrid model, designing double task heads including classification and positioning, adopting a hybrid loss function combining a Focal Loss function and a GIoU Loss function, and outputting a focus category and a bounding box; according to the invention, the identification accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of medical imaging technology, specifically to a deep learning-based medical image recognition method, system, electronic device, and medium. Background Technology

[0002] With the rapid development of medical imaging technology, clinical diagnosis has placed higher demands on the accuracy and efficiency of image analysis. Existing deep learning-based medical image recognition methods mainly have the following problems: traditional convolutional neural networks perform unevenly when processing lesions of different sizes, small lesions are easily overlooked, and the boundary recognition of large lesions is inaccurate; medical image data annotation costs are high and the sample size is limited, resulting in insufficient model generalization ability. Summary of the Invention

[0003] The purpose of this invention is to solve the above problems by designing a medical image recognition method and system based on deep learning.

[0004] The first aspect of this invention provides a deep learning-based medical image recognition method, which includes the following steps: The initial medical images are subjected to modal adaptive normalization and preprocessed with elastic deformation, random scaling and Gaussian noise to obtain the preprocessed medical images. A Swin-ResNeSt hybrid model was constructed based on preprocessed medical images. The bottom layer of the Swin-ResNeSt hybrid model extracts local features through the ResNeSt network, and the middle layer models global dependencies through the Swin Transformer network. Contrastive learning pre-training was completed on unlabeled medical images based on the MoCo v3 framework. An adaptive multi-scale attention module is added to the Swin-ResNeSt hybrid model, which includes a scale-aware weight calculation unit, a spatial adaptive adjustment unit, and a scale fusion unit. The design includes a dual-task head for classification and localization, employing a hybrid loss function combining Focal Loss and GIoU Loss to output lesion categories and bounding boxes.

[0005] Optionally, in a first implementation of the first aspect of the present invention, the step of performing modal adaptive normalization on the initial medical image and preprocessing it with elastic deformation, random scaling, and Gaussian noise to obtain a preprocessed medical image includes: The initial medical images are classified according to modality type, and a submodal adaptive normalization operation is performed for different modalities to obtain normalized images, wherein the modality type includes at least T1-weighted images and T2-weighted images of CT and MRI; A smooth displacement field is generated by Gaussian filtering, and the position of each pixel in the normalized image is adjusted according to the displacement field to perform elastic deformation processing. A scaling factor is randomly selected from the range of 0.8 to 1.2. The image after elastic deformation is randomly scaled according to the scaling factor. After scaling, the image is restored to its original size by filling. A Gaussian noise matrix with the same size as the image is generated based on the preset noise intensity. The Gaussian noise matrix is ​​added to the image pixel values ​​at corresponding positions. Gaussian noise is then added to the scaled image to obtain the preprocessed medical image.

[0006] Optionally, in a second implementation of the first aspect of the present invention, the construction of the Swin-ResNeSt hybrid model based on the preprocessed medical images includes: The first medical image is input into the bottom layer network. Through multi-layer convolutional operations with group attention, the local texture, edge and other detailed features of the image are extracted layer by layer, and multi-scale local feature maps are output. The ResNeSt network introduces a group attention mechanism in the residual block. The middle layer network is connected to the output of the bottom layer network, and the local feature map is divided into multiple non-overlapping square windows using the SwingTransformer network structure. Self-attention is calculated for the features within each window to obtain the local correlation features within the window. Attention calculation across windows is achieved through a sliding window mechanism, capturing global dependencies between different windows. The mid-level network outputs feature maps containing global contextual information. A pre-trained dual-branch network was built based on the MoCo v3 framework. Unlabeled medical images were input into the dual-branch network, and two different views of the same image were generated through random data augmentation. The feature vectors were obtained by processing the query branch and the key branch respectively. The InfoNCE loss of the feature vectors obtained by processing the query branch and key branch is calculated. The query branch parameters are updated according to the preset training rounds, learning rate and batch size to complete the contrastive learning pre-training.

[0007] Optionally, in a third implementation of the first aspect of the present invention, the dual-branch network includes a query branch and a key branch, wherein the query branch is a Swin-ResNeSt hybrid network, and the key branch adopts a network with the same structure as the query branch but with independent parameters.

[0008] Optionally, in a fourth implementation of the first aspect of the present invention, adding an adaptive multi-scale attention module to the Swin-ResNeSt hybrid model includes: A scale-aware weight calculation unit is connected to the output of the middle layer network of the Swin-ResNeSt hybrid model to collect multi-scale feature maps of the middle layer network output. For each scale of feature map, the features are first compressed through a 3×3 lightweight convolutional layer, and then the compressed features are input into the Sigmoid activation function to obtain the initial weights of each scale feature map. The spatial adaptive adjustment unit is activated to calculate the pixel-level spatial distribution entropy for each scale feature map, and the attention weights at the corresponding positions are dynamically adjusted according to the entropy value. The multi-scale feature maps after spatial adaptive adjustment are fused by a scale fusion unit. The feature map of each scale is multiplied by its corresponding initial weight and spatial adjustment weight to obtain a weighted feature map. All weighted feature maps are summed at pixel positions to generate a feature representation that integrates multi-scale information and has spatial adaptive attention.

[0009] Optionally, in a fifth implementation of the first aspect of the present invention, the design includes a dual-task head for classification and localization, employing a hybrid loss function combining the Focal Loss function and the GIoU Loss function to output lesion categories and bounding boxes, including: In the classification task head, the feature representation is input into a two-layer fully connected network, and the Sigmoid activation function is applied to the features output by the second layer to obtain the probability value of each category; The localization task head uses a fully convolutional network structure to perform multi-layer convolution operations on the feature representation, and outputs a feature map with the same size as the image. Each pixel position corresponds to a set of coordinate parameters, which are used to represent the bounding box of the lesion that may exist at the current position. The gradient descent algorithm is used to update the parameters of the Swin-ResNeSt hybrid model. After training, the medical image to be identified is input into the Swin-ResNeSt hybrid model. The lesion category is output through the classification task head, and the lesion bounding box is output through the localization task head.

[0010] Optionally, in a sixth implementation of the first aspect of the present invention, the first fully connected network outputs intermediate features of a preset dimension, and the second fully connected network outputs features corresponding to the number of lesion categories.

[0011] A second aspect of the present invention provides a deep learning-based medical image recognition system, the system comprising: The preprocessing module is used to perform modal adaptive normalization on the initial medical images and to preprocess them using elastic deformation, random scaling and Gaussian noise to obtain the preprocessed medical images. A model building module is used to construct a Swin-ResNeSt hybrid model based on preprocessed medical images. The bottom layer of the Swin-ResNeSt hybrid model extracts local features through the ResNeSt network, and the middle layer models global dependencies through the Swin Transformer network. The contrastive learning pre-training is completed on unlabeled medical images based on the MoCo v3 framework. Add a module to add an adaptive multi-scale attention module to the Swin-ResNeSt hybrid model, wherein the adaptive multi-scale attention module includes a scale-aware weight calculation unit, a spatial adaptive adjustment unit, and a scale fusion unit. The output module is used to design a dual-task head that includes classification and localization. It uses a hybrid loss function that combines the Focal Loss function and the GIoULoss function to output the lesion category and bounding box.

[0012] A third aspect of the present invention provides an electronic device comprising a memory and at least one processor, the memory storing instructions; the at least one processor invokes the instructions in the memory to cause the deep learning-based medical image recognition device to perform the steps of the deep learning-based medical image recognition method as described in any of the preceding claims.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of the deep learning-based medical image recognition method as described in any of the preceding claims.

[0014] The technical solution provided by this invention involves modal adaptive normalization of the initial medical image, followed by preprocessing with elastic deformation, random scaling, and Gaussian noise to obtain a preprocessed medical image. A Swin-ResNeSt hybrid model is then constructed based on the preprocessed medical image. An adaptive multi-scale attention module is added to the Swin-ResNeSt hybrid model, and a dual-task head including classification and localization is designed. A hybrid loss function combining Focal Loss and GIoU Loss functions is used to output the lesion category and bounding box. This invention proposes an adaptive multi-scale attention mechanism that can dynamically adjust the weights of features at different scales according to the image content, effectively solving the problem of imbalanced recognition of multi-size lesions, improving recognition accuracy, and meeting the recognition needs of complex clinical imaging environments. Attached Figure Description

[0015] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0016] Figure 1 A flowchart illustrating a deep learning-based medical image recognition method provided in an embodiment of the present invention; Figure 2 A schematic diagram of the structure of a deep learning-based medical image recognition system provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a deep learning-based medical image recognition device provided in an embodiment of the present invention. Detailed Implementation

[0017] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0018] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The flowchart of the deep learning-based medical image recognition method provided in this embodiment of the invention includes the following steps: Step 101: Perform modal adaptive normalization on the initial medical images, and preprocess them using elastic deformation, random scaling and Gaussian noise to obtain the preprocessed medical images; In this embodiment, the initial medical images are precisely classified according to modality, clearly distinguishing between CT images and T1-weighted and T2-weighted images of MRI. This is because the physical characteristics and signal meanings of images of different modalities are significantly different: the pixel values ​​of CT images are expressed in HU (Henry units), reflecting tissue density. It is necessary to extract the physiologically meaningful range of -1000 to 400 HU, covering key areas such as the lungs and soft tissues. Then, the values ​​in this range are converted into a standardized range of 0 to 1 through linear mapping to avoid interference from irrelevant density values. T1-weighted images of MRI focus on displaying anatomical structures, such as fat showing high signal. T2-weighted images are more likely to distinguish between edema and diseased tissue, such as fluid showing high signal. It is necessary to calculate the mean and standard deviation of all pixels in each modality. The mean is subtracted from each pixel value and then divided by the standard deviation to achieve grayscale consistency of images within the same modality. Finally, a normalized image is obtained, laying a unified foundation for subsequent feature extraction. Elastic deformation processing enhances the model's adaptability to organ position changes by simulating the natural movement of human organs, such as the lungs and brain, during breathing and heartbeat. Specifically, a continuous and smooth displacement field is first generated based on preset Gaussian filtering parameters, such as the filter kernel size and standard deviation. This displacement field ensures that the pixel movement amplitude changes gradually with spatial position, avoiding abrupt jumps. Then, based on the offset of each coordinate point in the displacement field, the new position of the corresponding pixel in the normalized image is recalculated. For pixels that exceed the original image range after displacement, edge pixel values ​​are used to fill them. For pixels in overlapping areas, interpolation methods, such as bilinear interpolation, are used to determine the final value, so that the deformed image retains the original anatomical structure correlation and has the diversity of movement states. Random scaling is designed to adapt to the resolution differences of images acquired by different medical devices, such as the different slice thicknesses and fields of view of different CT devices, thereby improving the robustness of the model across devices. During operation, a scaling factor is randomly selected from the scale range of 0.8 to 1.2. This range covers the resolution fluctuation range commonly seen in clinical practice and avoids structural distortion caused by over-scaling. After scaling the elastically deformed image proportionally according to the selected factor, an edge filling method is used, such as copying the edge pixel values ​​of the image and expanding them in all directions to restore the image to its original size. This ensures that the scaled image maintains the structural integrity of key lesions while its spatial dimension is consistent with the original input. Adding Gaussian noise is to simulate potential equipment noise in clinical images, such as quantum noise in low-dose CT and electromagnetic interference noise in MRI, to enhance the model's noise resistance. First, based on a preset noise intensity, such as the noise standard deviation, typically set to 0.01~0.03, a Gaussian noise matrix with the same size as the image is generated, matching the typical noise level of clinical images. The pixel values ​​of this matrix follow a normal distribution with a mean of 0 and a preset standard deviation. Then, the noise matrix is ​​added to the scaled image pixel by pixel, so that the noise is evenly distributed throughout the image, resulting in the preprocessed medical image. This processing preserves the original lesion features of the image while artificially introducing controllable noise, forcing the model to learn more robust lesion recognition features and reducing the interference of noise on the recognition results.

[0019] Step 102: Construct a Swin-ResNeSt hybrid model based on the preprocessed medical images. The Swin-ResNeSt hybrid model extracts local features through the ResNeSt network at the bottom layer and models global dependencies through the Swin Transformer network in the middle layer. The contrastive learning pre-training is completed on unlabeled medical images based on the MoCo v3 framework. In this embodiment, the first medical image is input into the ResNeSt network, the bottom layer of the Swin-ResNeSt hybrid model. This network innovatively introduces a group attention mechanism on the basis of traditional residual blocks. First, the feature channels output by the residual blocks are divided into several non-overlapping groups. Within each group, channel attention weights are calculated through fully connected layers and softmax activation functions. The weight values ​​reflect the importance of the channel features to lesion identification. Then, through multi-layer convolutional operations with group attention, each convolutional kernel size is 3×3, and the stride increases with the network depth to achieve downsampling. Local features at different levels in the image are extracted layer by layer: shallow networks capture pixel-level textures such as grayscale changes at the edge of lesions, middle networks extract local structures such as the circular outline of small nodules, and deep networks aggregate more abstract detailed features such as the density difference between lesions and surrounding tissues. Finally, local feature maps at multiple scales such as 1 / 2, 1 / 4, 1 / 8, and 1 / 16 are output. The middle-layer SwinTransformer network is connected to the output of the bottom-layer network. First, the multi-scale local feature map is divided into multiple non-overlapping square windows according to a preset window size, such as 7×7 pixels. The choice of window size takes into account both computational efficiency and local feature integrity to ensure that small lesions are completely contained within a single window. For the features within each window, a query, key, and value matrix is ​​generated through linear mapping. The similarity between the query and the key is calculated to obtain an attention score. After softmax normalization, the score is weighted and summed with the value matrix to complete the self-attention calculation within the window. This process can capture the spatial correlation between pixels within the window, such as the continuous texture of the lesion edge, and finally obtain the local correlation features of each window, enhancing the feature expression of small-scale lesion structures. Cross-window attention computation is achieved through a sliding window mechanism: the segmented windows are slid horizontally and vertically at a preset step size, such as 1 / 2 of the window size, so that adjacent windows overlap; for the features of the overlapping areas, the attention scores between different windows are recalculated, that is, the correlation strength between the features of one window and the features of adjacent windows. Through this cross-window interaction, the spatial limitations of a single window are broken, and the global dependencies between different windows are captured, such as the positional association between lung nodules and blood vessels, and the spatial distribution of brain tumors and ventricles; after multi-layer cross-window attention computation, the middle layer network outputs a feature map that integrates local details and global context, which not only preserves the fine structural information of the lesion, but also includes its spatial location information in the whole organ; A pre-trained dual-branch network was built based on the MoCo v3 framework. The query branch directly adopted the Swin-ResNeSt hybrid network constructed above, while the key branch adopted a network with the same structure as the query branch but independent parameters. The key branch parameters did not participate in gradient updates, but were only associated with the query branch through a momentum mechanism. A large number of unlabeled medical images, such as unlabeled CT and MRI sequences, were input into the dual-branch network. Different data augmentation strategies, such as different degrees of elastic deformation, scaling, and noise injection, were randomly applied to each image to generate two different views, which were then input into the query branch and the key branch respectively. The two branches output high-dimensional feature vectors through their respective feature extraction processes: low-level local features + mid-level global features. The query branch features were used for subsequent loss calculation, and the key branch features served as a benchmark. The InfoNCE loss of the output feature vectors of the query branch and the key branch is calculated as follows: Two view features of the same image are regarded as positive sample pairs and compared with negative sample pairs of view features of other images in the batch. The similarity probability of the positive sample pairs is calculated by the softmax function. The loss function is constructed with the goal of maximizing the positive sample probability and minimizing the negative sample probability. According to the preset training parameters, such as 100 training epochs, initial learning rate 5e-4, and batch size 64, only the network parameters of the query branch are updated. The key branch parameters are slowly tracked by the query branch parameters through a momentum coefficient of 0.999. After complete pre-training, the query branch, i.e., the Swin-ResNeSt hybrid network, has learned the general feature patterns of medical images from unlabeled data, such as the anatomical structure distribution of organs and the common features of lesions.

[0020] Step 103: Add an adaptive multi-scale attention module to the Swin-ResNeSt hybrid model, wherein the adaptive multi-scale attention module includes a scale-aware weight calculation unit, a spatial adaptive adjustment unit, and a scale fusion unit. In this embodiment, a scale-aware weight calculation unit is connected to the output of the middle layer network of the Swin-ResNeSt hybrid model. The primary task of this unit is to collect multi-scale feature maps output by the middle layer network. These feature maps originate from the global dependency modeling of the lower-level local features by the middle layer SwinTransformer network, covering four typical resolutions: 1 / 2, 1 / 4, 1 / 8, and 1 / 16. Among them, high-resolution features... Figure 1 / 2, 1 / 4 retain more detailed information such as lesion edges and textures, low-resolution features Figure 1 / 8 and 1 / 16 contain global association information between the lesion and surrounding tissues; the unit stores multi-scale feature maps in resolution order through a preset feature receiving interface, providing structured input for subsequent weight calculations at different scales, and avoiding weight allocation deviations caused by feature confusion at different scales. For each resolution feature map, feature compression is first performed: a 3×3 lightweight convolutional layer is used to perform convolution calculations on the feature map. The number of convolutional kernels in this convolutional layer is only 1 / 4 of the number of channels in the corresponding feature map. For example, if the original feature map contains 256 channels, the convolutional layer is set to 64 convolutional kernels. While reducing the amount of computation, the 3×3 receptive field captures the correlation information of the local neighborhood of the feature map, realizing the dimensionality reduction of the feature channels and the filtering of redundant information. Then, the compressed single-channel feature map is input into the Sigmoid activation function. This function maps the feature values ​​to the range of 0 to 1. The mapping result is the initial weight of the feature map at the corresponding scale. The closer the weight value is to 1, the higher the contribution of the feature map at that scale to lesion recognition. The closer it is to 0, the lower the contribution. This initially realizes the distinction of the importance of features at different scales. After activating the spatial adaptive adjustment unit, the unit first performs pixel-level spatial distribution entropy calculation on the feature map at each scale: for each pixel in the feature map, the distribution frequency of feature values ​​in the 8-neighborhood region is statistically analyzed, and then the entropy value of the pixel position is calculated using the information entropy formula such as Shannon entropy. The higher the entropy value, the more drastic the feature changes around the pixel, which is likely to be a densely populated area such as the edge or core of the lesion; the lower the entropy value, the more flat the features in the region, which is mostly normal tissue or background area. Based on the entropy value results, the unit dynamically adjusts the attention weight of each pixel position: the entropy value is divided into three intervals of high, medium and low according to a preset threshold. The weight of the high entropy interval is increased to 1.5 to 2 times the original weight, the weight of the medium entropy interval is kept at the original weight, and the weight of the low entropy interval is reduced to 0.3 to 0.5 times the original weight, so that the attention is more focused on the key spatial regions related to the lesion. The scale fusion unit first retrieves the initial weights corresponding to each scale feature map from the scale-aware weight calculation unit and the spatial adjustment weights from the spatial adaptive adjustment unit. Then, according to the calculation logic of feature map × initial weight × spatial adjustment weight, the feature map of each scale is double-weighted: the initial weights select high-contribution features from the scale dimension, and the spatial adjustment weights strengthen key region features from the pixel dimension. The combination of the two achieves double suppression of background regions with low contribution of ineffective features. For example, in the 1 / 2 resolution feature map with high initial weight, high-entropy pixels at the edge of the lesion will be given a higher comprehensive weight, while low-entropy background pixels in the 1 / 16 resolution feature map with low initial weight will be given a very low comprehensive weight. Finally, the weighted feature map of each scale is obtained, ensuring that effective features are highlighted and preserved. Pixel-level addition fusion is performed on all weighted multi-scale feature maps: the weighted feature maps at resolutions of 1 / 2, 1 / 4, 1 / 8, and 1 / 16 are upsampled to a uniform 1 / 4 resolution using bilinear interpolation to balance detail preservation and computational efficiency. Then, they are added one by one according to the corresponding pixel position. The final value of each pixel is the sum of the values ​​of the weighted feature maps at that position. This fusion method can aggregate effective information from different scales, such as high-resolution detailed features and low-resolution global features, into the same feature map. At the same time, due to the double weighting in the early stage, the interference of background and redundant information on the fusion result is avoided. The final generated feature representation contains multi-scale lesion information and has spatially adaptive attention focusing characteristics.

[0021] Step 104: Design a dual-task head that includes classification and localization, and use a hybrid loss function that combines Focal Loss and GIoU Loss functions to output lesion categories and bounding boxes.

[0022] In this embodiment, the feature representation in the classification task head is input into a two-layer fully connected network. The first fully connected network outputs intermediate features of a preset dimension, and the second fully connected network outputs features corresponding to the number of lesion categories. The Sigmoid activation function is applied to the features output by the second layer to obtain the probability value of each category. In the localization task head, a fully convolutional network structure is used to perform multi-layer convolution operations on the feature representation, outputting a feature map with the same size as the image. Each pixel position corresponds to a set of coordinate parameters used to represent the bounding box of the lesion that may exist at the current position. The gradient descent algorithm is used to update the parameters of the Swin-ResNeSt hybrid model. After training, the medical image to be identified is input into the Swin-ResNeSt hybrid model. The lesion category is output by the classification task head, and the lesion bounding box is output by the localization task head.

[0023] Please see Figure 2 A schematic diagram of the structure of a deep learning-based medical image recognition system provided in this embodiment of the invention. The system includes: The preprocessing module is used to perform modal adaptive normalization on the initial medical images and to preprocess them using elastic deformation, random scaling and Gaussian noise to obtain the preprocessed medical images. A model building module is used to construct a Swin-ResNeSt hybrid model based on preprocessed medical images. The bottom layer of the Swin-ResNeSt hybrid model extracts local features through the ResNeSt network, and the middle layer models global dependencies through the Swin Transformer network. The contrastive learning pre-training is completed on unlabeled medical images based on the MoCo v3 framework. Add a module to add an adaptive multi-scale attention module to the Swin-ResNeSt hybrid model, wherein the adaptive multi-scale attention module includes a scale-aware weight calculation unit, a spatial adaptive adjustment unit, and a scale fusion unit. The output module is used to design a dual-task head that includes classification and localization. It uses a hybrid loss function that combines the Focal Loss function and the GIoULoss function to output the lesion category and bounding box.

[0024] Figure 3 This is a schematic diagram of the structure of a deep learning-based medical image recognition device 300 provided in an embodiment of the present invention. The deep learning-based medical image recognition device 300 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the deep learning-based medical image recognition device 300. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the deep learning-based medical image recognition device 300 to implement the method provided in the above embodiment.

[0025] The deep learning-based medical image recognition device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The structure of the deep learning-based medical image recognition device shown does not constitute a limitation on the computer device provided by the present invention. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0026] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the various steps of the deep learning-based medical image recognition method provided in the above embodiments.

[0027] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described equipment or apparatus / unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0028] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0029] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A medical image recognition method based on deep learning, characterized by, The method comprises the following steps: The initial medical image is subjected to adaptive normalization by modality, and is preprocessed by elastic deformation, random scaling and Gaussian noise to obtain a preprocessed medical image; A Swin-ResNeSt hybrid model is constructed based on the preprocessed medical image, wherein the bottom layer of the Swin-ResNeSt hybrid model extracts local features through a ResNeSt network, the middle layer models global dependence through a Swin Transformer network, and contrast learning pretraining is completed on unlabeled medical images based on a MoCo v3 framework; An adaptive multi-scale attention module is added to the Swin-ResNeSt hybrid model, wherein the adaptive multi-scale attention module comprises a scale perception weight calculation unit, a spatial adaptive adjustment unit and a scale fusion unit; A double-task head comprising classification and positioning is designed, a hybrid loss function combining a Focal Loss function and a GIoU Loss function is adopted, and a lesion category and a bounding box are output. 2.The deep learning-based medical image recognition method of claim 1, wherein, The adaptive normalization by modality and the preprocessing of the initial medical image by elastic deformation, random scaling and Gaussian noise to obtain the preprocessed medical image comprise: The initial medical image is classified by modality type, and a modality-specific adaptive normalization operation is performed on different modalities to obtain a normalized image, wherein the modality types at least include CT, MRI T1 weighted image and T2 weighted image; A smooth displacement field is generated by Gaussian filtering, and the position of each pixel in the normalized image is adjusted according to the displacement field to perform elastic deformation processing; A scaling coefficient is randomly selected from a ratio range of 0.8-1.2, and the image after elastic deformation is randomly scaled by the scaling coefficient, and the image is restored to the original size after scaling by padding; A Gaussian noise matrix with the same size as the image is generated according to a preset noise intensity, the Gaussian noise matrix and the image pixel value are added at the corresponding positions, the Gaussian noise is added to the scaled image, and the preprocessed medical image is obtained. 3.The deep learning-based medical image recognition method of claim 1, wherein, The Swin-ResNeSt hybrid model is constructed based on the preprocessed medical image, comprising: The first medical image is input into the bottom layer network, and the local texture, edge detail features of the image are extracted layer by layer through multi-layer convolution operation with grouping attention, and a multi-scale local feature map is output, wherein the ResNeSt network introduces a grouping attention mechanism in the residual block; The output end of the bottom layer network is connected to the middle layer network, and the SwinTransformer network structure is used to divide the local feature map into a plurality of non-overlapping square windows; self-attention calculation is performed on the features in each window to obtain local correlation features in the window; Attention calculation across windows is realized through a sliding window mechanism to capture global dependence between different windows, and the middle layer network outputs a feature map containing global context information; The pre-training dual-branch network is built based on the MoCo v3 framework, the unlabeled medical image is input into the dual-branch network, two different views of the same image are generated through random data augmentation, and feature vectors are obtained through the query branch and the key branch; The InfoNCE loss of the feature vectors obtained by the query branch and the key branch is calculated, the query branch parameters are updated according to the preset training round, learning rate and batch size, and the contrast learning pre-training is completed. 4.The deep learning-based medical image recognition method of claim 3, wherein, The dual-branch network includes a query branch and a key branch, wherein the query branch is a Swin-ResNeSt hybrid network, and the key branch adopts a network with the same structure as the query branch but independent parameters. 5.The deep learning-based medical image recognition method of claim 1, wherein, The adaptive multi-scale attention module is added in the Swin-ResNeSt hybrid model, including: A scale perception weight calculation unit is connected to the output end of the middle layer network of the Swin-ResNeSt hybrid model, and the multi-scale feature maps output by the middle layer network are collected; For each scale feature map, first, a 3*3 lightweight convolution layer is used for feature compression, and then the compressed features are input into a Sigmoid activation function to obtain the initial weight of each scale feature map; A spatial adaptive adjustment unit is started, and the spatial distribution entropy of each scale feature map is calculated, and the attention weight of the corresponding position is dynamically adjusted according to the entropy value; Through the scale fusion unit, the multi-scale feature maps adjusted by the spatial adaptive adjustment unit are fused, and each scale feature map is multiplied by its corresponding initial weight and spatial adjustment weight to obtain a weighted feature map; All weighted feature maps are added according to the pixel position to generate a feature representation that fuses multi-scale information and has spatial adaptive attention. 6.The deep learning-based medical image recognition method of claim 5, wherein, The design includes a dual-task head for classification and positioning, adopts a hybrid loss function combining Focal Loss function and GIoU Loss function, and outputs lesion categories and bounding boxes, including: In the classification task head, the feature representation is input into a two-layer fully connected network, and a Sigmoid activation function is applied to the output features of the second layer to obtain probability values of each category; In the positioning task head, a full convolution network structure is used to perform multi-layer convolution operation on the feature representation, and a feature map with the same size as the image is output, wherein each pixel position corresponds to a group of coordinate parameters for representing the lesion bounding box existing at the current position; The gradient descent algorithm is used to update the parameters of the Swin-ResNeSt hybrid model, and after training, the medical image to be identified is input into the Swin-ResNeSt hybrid model, the lesion category is output through the classification task head, and the lesion bounding box is output through the positioning task head. 7.The deep learning-based medical image recognition method of claim 6, wherein, The first layer fully connected network outputs intermediate features of a preset dimension, and the second layer fully connected network outputs features corresponding to the number of lesion categories.

8. A medical image recognition system based on deep learning, characterized by, The system comprises: A preprocessing module is configured to perform modality-adaptive normalization on the initial medical image, and perform preprocessing such as elastic deformation, random scaling and Gaussian noise to obtain a preprocessed medical image; The model construction module is configured to construct a Swin-ResNeSt hybrid model based on the preprocessed medical image, wherein the Swin-ResNeSt hybrid model extracts local features through a ResNeSt network at a bottom layer, models global dependence through a Swin Transformer network at a middle layer, and completes contrast learning pre-training on unannotated medical images based on a MoCo v3 framework; The adding module is configured to add an adaptive multi-scale attention module to the Swin-ResNeSt hybrid model, wherein the adaptive multi-scale attention module comprises a scale perception weight calculation unit, a spatial adaptive adjustment unit, and a scale fusion unit; The output module is configured to design a double-task head including classification and positioning, adopt a hybrid loss function combining a Focal Loss function and a GIoU Loss function, and output a lesion category and a bounding box.

9. An electronic device, comprising: The electronic device includes a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to enable the deep learning-based medical image recognition device to perform the steps of the deep learning-based medical image recognition method according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon instructions, the instructions comprising, The instructions are executed by the processor to implement the steps of the deep learning-based medical image recognition method according to any one of claims 1-7.

Images