A medical image analysis and processing system based on image analysis

Abstract

The present invention discloses a medical image analysis and processing system based on image analysis, which relates to the technical field of medical image analysis, and specifically includes an image acquisition module, a data preprocessing module, an image segmentation module, a feature extraction module, a decision analysis module and a result output module. The image segmentation module segments the lesion area image and the organ area image from the preprocessed medical image data through a fusion model, and the fusion model is specifically a fusion model of multi-scale U-Net, PSPNet, hybrid CNN-Transformer and multi-task learning; the decision analysis module uses PU semi-supervised learning to combine positive samples and unlabeled data to train the classification model, and implements a semi-supervised disease classification strategy through the classification model. The present invention uses PU semi-supervised learning to effectively utilize limited labeled data and a large amount of unlabeled data, thereby improving the ability to extract image features and the accuracy of classification and segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image analysis, and in particular to a medical image analysis and processing system based on image analysis. Background Art

[0002] Medical images are images that reflect the internal structure of the human body. Compared with other images, they are usually highly complex and diverse, and generally contain a large amount of grayscale information and detail information. At present, medical image processing tasks mainly focus on four aspects: image detection, image segmentation, image registration, and image fusion.

[0003] Image processing technology is used to analyze and process medical images to achieve position detection, segmentation extraction, 3D reconstruction and 3D display of human organs, soft tissues and lesions. It can perform qualitative and even quantitative analysis on regions of interest (ROI), thereby greatly improving the efficiency, accuracy and reliability of clinical diagnosis. It can also play an important auxiliary role in medical teaching, surgical planning, surgical simulation and various medical research.

[0004] At present, the objects of medical image segmentation processing are mainly images of various cells, tissues, and organs. Although new segmentation methods are constantly proposed, the results are not perfect. It is difficult for any existing single image segmentation algorithm to achieve satisfactory results in medical image segmentation. The research focus of segmentation methods is mainly on automatic, accurate, fast, adaptive, robust, multi-modal fusion and other directions.

[0005] Artificial intelligence, such as deep learning methods such as CNN (Convolutional Neural Networks), has been widely used in medical image analysis and processing. However, CNN requires a large amount of labeled training data, which may be difficult to meet in the medical field where expert annotation is expensive and diseases are scarce. Secondly, training deep CNN requires a lot of computing and memory resources, otherwise the training process will be very time-consuming. In addition, the training process of deep CNN is complicated by overfitting and convergence problems, which usually requires repeated adjustments to the network's framework structure or learning parameters to ensure that all layers learn at a comparable speed. Summary of the invention

[0006] In view of the problems of inaccurate image segmentation results, scarce labeled data, and processing accuracy to be improved in medical image processing, the present invention proposes a medical image analysis and processing system based on image analysis, which adopts a fusion model and uses semi-supervised learning to effectively utilize limited labeled data and a large amount of unlabeled data, thereby improving the ability to extract image features and the accuracy of classification and segmentation.

[0007] In order to achieve the above object, the present invention is implemented by the following technical solutions:

[0008] A medical image analysis and processing system based on image analysis, the system comprising:

[0009] An image acquisition module, connected to the medical imaging device interface, for automatically receiving or manually uploading raw medical image data;

[0010] A data preprocessing module, used to preprocess the collected medical image data by using image processing technology;

[0011] An image segmentation module, used to segment lesion area images and organ area images from preprocessed medical image data through a fusion model, wherein the fusion model is specifically a fusion model of multi-scale U-Net, PSPNet, hybrid CNN-Transformer and multi-task learning;

[0012] A feature extraction module is used to extract key feature information again from the segmented lesion region image and / or organ region image, wherein the key feature information includes shape feature, size feature, position feature, texture feature, density and intensity feature of the lesion region;

[0013] The decision analysis module uses PU semi-supervised learning to train the classification model based on the extracted key feature information and combines positive samples and unlabeled data, and implements the semi-supervised disease classification strategy through the classification model;

[0014] The result output module is used to display the classification results of the classification model through a graphical user interface and store them in a database or a medical health record system.

[0015] As a preferred solution of the present invention, the image acquisition module further includes an intelligent image quality detection unit and an expansion unit;

[0016] The intelligent image quality detection unit is used to integrate an image quality detection algorithm to automatically detect and filter low-quality images that are blurred or abnormally exposed;

[0017] The expansion unit is used to automatically adapt to the interface protocols of different medical imaging devices, and realize image acquisition of medical imaging devices of various models through a universal interface.

[0018] As a preferred solution of the present invention, the data preprocessing module includes an adaptive filtering unit, an image enhancement unit and an image standardization unit;

[0019] The adaptive filtering unit is used to perform denoising on the medical image data by adaptively adjusting the filtering parameters according to the local characteristics of the medical image data through an adaptive Gaussian-median filter;

[0020] The image enhancement unit is used to integrate contrast-limited adaptive histogram equalization and multi-scale Retinex algorithm to enhance the contrast of medical image data;

[0021] The image standardization unit is used to perform standardization processing on cross-device and cross-modality medical image data, and through gamma correction and intensity normalization, the medical image data from different sources have consistent brightness and contrast characteristics.

[0022] As a preferred solution of the present invention, the adaptive Gaussian-median filter specifically includes:

[0023] Calculate the mean and variance of the local area of ​​the medical image, and adaptively adjust the filter parameters according to the local image characteristics. The formula is:

[0024]

[0025] In the formula, μ i,j is the mean of the local area around pixel (i, j), k is the local window radius, and the window size is (2k+1)×(2k+1);

[0026] u represents the offset of the pixel in the window relative to the center pixel i in the row direction, v represents the offset of the pixel in the window relative to the center pixel j in the column direction; I(i+u,j+v) is the pixel value of the medical image I at position (i+u,j+v); σ 2 i,j is the variance of the local area around pixel (i, j);

[0027] Dynamically adjust the standard deviation of the Gaussian filter based on local statistical characteristics:

[0028] σ adaptive (i,j)=α·σ i,j +β;

[0029] In the formula, σ adaptive (i, j) represents the standard deviation of the adaptively adjusted Gaussian filter for pixel (i, j); α is the scaling factor; σ i,j is the standard deviation of the local area around pixel (i, j); β is the offset;

[0030] Apply an adaptive Gaussian filter for convolution:

[0031]

[0032] In the formula, G adaptive (x,y;σ adaptive ) is the adaptive Gaussian kernel, x and y are the position coordinates relative to the center of the filter, σ adaptiveis the standard deviation of the Gaussian filter after adaptive adjustment; I blur (i, j) is the value of pixel (i, j) after Gaussian filtering; I(i+k, j+l) represents the pixel value of the medical image at position (i+k, j+l); k and l are the row and column offsets, respectively; K and L are the radii of the Gaussian kernel; G adaptive (k,l;σ adaptive (i,j)) is the adaptive Gaussian kernel value;

[0033] According to the local statistical characteristics, the window size of the median filter is dynamically adjusted:

[0034]

[0035] Where W adaptive (i, j) is the adaptively adjusted median filter window size for pixel (i, j); W max , W min are the maximum window size and minimum window size of the median filter respectively; σ max is the maximum value of the standard deviation of all local regions in the medical image;

[0036] Apply an adaptive median filter:

[0037] I denoised (i,j)=median({I blur (i+k,j+l)|-W adaptive / 2≤k,l≤W adaptive / 2});

[0038] In the formula, I denoised (i, j) is the value of pixel (i, j) after median filtering; I blur (i+k,j+l) is the pixel value of the medical image at position (i+k,j+l) after Gaussian filtering; W adaptive The window size of the median filter after adaptive adjustment.

[0039] As a preferred solution of the present invention, the hybrid CNN-Transformer extracts local features through CNN, captures global and long-distance dependency features through Transformer, and adopts a dual decoder structure, one decoder is used for segmentation of lesion areas, and the other decoder is used for segmentation of organ areas. An attention gate is introduced in each decoder to simultaneously process the segmentation tasks of lesions and organs.

[0040] As a preferred solution of the present invention, the fusion model specifically includes:

[0041] The encoder part combines multi-scale U-Net and hybrid CNN-Transformer. In the encoder part of U-Net, the architecture of multi-scale U-Net is used to extract multi-resolution features through convolutional layers of different scales to perform multi-scale feature extraction;

[0042] The decoder part adopts a hybrid CNN-Transformer dual decoder structure, which serves as a lesion decoder and an organ decoder respectively. The lesion decoder is used for the segmentation of lesion areas, and adopts a decoder architecture based on a multi-scale U-Net to restore spatial resolution through layer-by-layer upsampling. The organ decoder is used for the segmentation of organ areas, and uses the pyramid pooling module of PSPNet to capture multi-scale contextual information.

[0043] Using a multi-task learning framework, the encoder part fuses multi-scale U-Net and hybrid CNN-Transformer as a shared encoder to extract common features of the input image; the lesion decoder and organ decoder are used as independent decoders to process the segmentation tasks of lesion areas and organ areas respectively; during the training process, the multi-task loss function is used to simultaneously optimize the segmentation tasks of lesion areas and organ areas, and the weights are adjusted according to the importance of the tasks.

[0044] As a preferred solution of the present invention, in the fusion model encoder part, a multi-scale convolution layer is used in each layer of the encoder to extract features of different resolutions. The formula is:

[0045] x b =Conv a×a (x b -1);

[0046] In the formula, x b is the feature map of layer b, Conv a×a Represents the convolution operation of a×a;

[0047] Based on multi-scale feature extraction, a Transformer module is inserted after each U-Net encoder layer, and the Transformer layer is used to capture global features, including:

[0048] Divide the input image into non-overlapping patches x v :

[0049]

[0050] Where S is the size of each patch, N is the total number of patches, X is the number of channels, and R represents the real number space.

[0051] Each patch is transformed by a linear matrix Mapping to the embedding space, we get the embedding vector z0:

[0052] z0=[x v1 E,x v2 E,...,x vN E];

[0053] Where Y is the dimension of the embedding space; x v1 Indicates the first patch, x v2 Indicates the second patch, x vN Indicates the Nth patch;

[0054] A multi-head self-attention mechanism is applied in the Transformer layer to capture global dependencies:

[0055] z ′ g =MSA(LN(z g -1))+z g -1;

[0056] In the formula, z g is the embedding vector of the Transformer encoder at layer g; LN represents layer normalization, MSA represents the multi-head self-attention mechanism; z′ g Represents the intermediate representation after multi-head self-attention mechanism and layer normalization.

[0057] As a preferred solution of the present invention, the lesion decoder uses convolutional layers and upsampling layers to restore spatial resolution layer by layer, and the formula is:

[0058] d g1 =ResB(Concatenate[AttGate(x g ,UpSample(x g+1 )),UpSample(x g+1 )]);

[0059] Where, d g1 represents the output feature map of the lesion decoder at layer g; x g represents the input feature map of the decoder at layer g, x g+1 Represents the input feature map of the decoder at the g+1 layer; ResB represents the residual block; AttGate represents the attention gate; UpSample represents upsampling; Concatenate represents the concatenation operation;

[0060] The organ decoder uses the pyramid pooling module of PSPNet to capture multi-scale contextual information, the formula is:

[0061] d g2 =PSP(ResB(Concatenate[AttGate(x g,UpSample(x g+1 )),UpSample(x g+1 )]));

[0062] Where, d g2 represents the output feature map of the organ decoder at the gth layer; PSP represents the pyramid pooling module in PSPNet.

[0063] As a preferred solution of the present invention, in the multi-task learning framework, a multi-task loss function L is used. hybrid , combined with cross entropy loss and Dice loss for optimization:

[0064]

[0065] Where, L LES-CE and L LES-Dice are the cross entropy loss and Dice loss of the diseased branch, L O-CE is the CE loss of the organ branch, ω, γ is the weight parameter.

[0066] As a preferred solution of the present invention, the method of using PU semi-supervised learning to train the classification model in combination with positive samples and unlabeled data specifically includes:

[0067] The PU dataset is constructed by extracting key feature information, and h data are randomly selected and manually labeled as positive samples, and the rest are unlabeled data;

[0068] Construct a classification model based on generative adversarial networks, use a classifier instead of a generator to determine the probability that any data in the unlabeled data is a positive sample, and pass the data determined as a positive sample to the discriminator, and use the discriminator to determine the authenticity of the data as a positive sample;

[0069] The objective function of the classification model is constructed based on the Holder divergence, and the objective function f(D, C) is expressed as:

[0070]

[0071] Where C is the classifier, D is the discriminator; M is the number of samples in the PU dataset, M0 is the number of unlabeled data in the PU dataset; P m is the probability distribution of positive samples in the PU dataset, D m is the discriminant distribution of the mth data in the PU dataset; H is the Holder divergence; δ is a balance hyperparameter used to adjust the weights between different loss terms; is the discriminant distribution of the mth data in the unlabeled data, is the probability distribution of the classifier for the mth data in the unlabeled data; Represents the inverse distribution of the classifier, which is the supplementary distribution of the classifier for the mth data in the unlabeled data.

[0072] The beneficial effects of the present invention are as follows: by using a multi-scale convolutional layer to extract features of different resolutions in each layer of the encoder, various details and global information in the image can be captured more comprehensively, thereby improving the representation ability and analysis accuracy of the image features; a Transformer module is inserted after each U-Net encoder layer, and a multi-head self-attention mechanism is used to capture global dependencies, enhance the model's understanding of the relationship between distant pixels in the image, and further improve the model's expressiveness and accuracy; the lesion decoder uses convolutional layers and upsampling layers to restore the spatial resolution layer by layer, ensuring that the detail information of the original image is retained as much as possible during the decoding process, and improving the accuracy of the segmentation result; the organ decoder uses the pyramid pooling module of PSPNet to capture multi-scale contextual information, helping the model to better understand the image content at different scales and improve the accuracy of organ segmentation. The multi-task loss function is used in combination with the cross entropy loss and the Dice loss for optimization, and the lesion branch and the organ branch can be optimized at the same time, so that the model can achieve better performance when processing different tasks, and improve the overall analysis effect. Semi-supervised learning is used to effectively utilize limited labeled data and a large amount of unlabeled data, improve the image feature extraction ability and the accuracy of classification and segmentation, and significantly improve the performance and accuracy of the image analysis and processing system. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0074] in:

[0075] Figure 1 A schematic diagram of a modular structure of a system in an embodiment of the present invention;

[0076] Figure 2 Schematic diagram of the fusion model architecture in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0078] like Figure 1 , Figure 2 As shown, it is an embodiment of the present invention, which provides a medical image analysis and processing system based on image analysis, including an image acquisition module, a data preprocessing module, an image segmentation module, a feature extraction module, a decision analysis module and a result output module.

[0079] Image acquisition module: connected to the interface of medical imaging equipment, used to automatically receive or manually upload original medical image data;

[0080] A real-time image acquisition function can be added to achieve real-time acquisition and processing of medical image data through a high-speed data transmission interface (such as USB 3.0 or GigE) to reduce latency; support wireless transmission technology (such as WiFi or Bluetooth) to achieve wireless transmission of medical image data, improve the flexibility and convenience of the system, and adapt to more application scenarios.

[0081] In one of the embodiments, the image acquisition module further includes an intelligent image quality detection unit and an expansion unit;

[0082] The intelligent image quality detection unit is used to integrate image quality detection algorithms to automatically detect and filter low-quality images that are blurred or have abnormal exposure, so as to ensure the high quality of input data and improve the accuracy of subsequent processing;

[0083] The expansion unit supports a variety of medical imaging devices, including MRI, CT, ultrasound, X-ray, etc. It is used to automatically adapt to the interface protocols of different medical imaging devices, realize image acquisition of various models of medical imaging devices through a universal interface, and improve the versatility and response speed of the system through multi-modal imaging compatibility and real-time data processing.

[0084] Data preprocessing module: used to preprocess the collected medical image data through image processing technology, including an adaptive filtering unit, an image enhancement unit and an image standardization unit;

[0085] The adaptive filtering unit is used to adaptively adjust the filtering parameters according to the local characteristics of the medical image data through an adaptive Gaussian-median filter, perform denoising on the medical image data, improve the denoising effect and detail retention ability, and enhance the clarity of the image;

[0086] In one embodiment, the adaptive Gaussian-median filter specifically includes:

[0087] Calculate the mean and variance of the local area of ​​the medical image, and adaptively adjust the filter parameters according to the local image characteristics. The formula is:

[0088]

[0089]

[0090] In the formula, μ i,j is the mean of the local area around pixel (i, j), k is the local window radius, and the window size is (2k+1)×(2k+1); u represents the offset of the pixel in the window relative to the center pixel i in the row direction, and v represents the offset of the pixel in the window relative to the center pixel j in the column direction; I(i+u,j+v) is the pixel value of medical image I at position (i+u,j+v); σ 2 i,j is the variance of the local area around pixel (i, j);

[0091] Dynamically adjust the standard deviation of the Gaussian filter based on local statistical characteristics:

[0092] σ adaptive (i,j)=α·σ i,j +β;

[0093] In the formula, σ adaptive (i, j) represents the standard deviation of the adaptively adjusted Gaussian filter for pixel (i, j); α is the scaling factor; σ i,j is the standard deviation of the local area around pixel (i, j); β is the offset;

[0094] Apply an adaptive Gaussian filter for convolution:

[0095]

[0096] In the formula, G adaptive (x,y;σ adaptive ) is the adaptive Gaussian kernel, x and y are the position coordinates relative to the center of the filter, σ adaptive is the standard deviation of the Gaussian filter after adaptive adjustment; I blur (i, j) is the value of pixel (i, j) after Gaussian filtering; I(i+k, j+l) represents the pixel value of the medical image at position (i+k, j+l); k and l are the row and column offsets, respectively; K and L are the radii of the Gaussian kernel; G adaptive (k,l;σ adaptive (i,j)) is the adaptive Gaussian kernel value;

[0097] According to the local statistical characteristics, the window size of the median filter is dynamically adjusted:

[0098]

[0099] Where W adaptive (i, j) is the adaptively adjusted median filter window size for pixel (i, j); W max, W min are the maximum window size and minimum window size of the median filter respectively; σ max is the maximum value of the standard deviation of all local regions in the medical image;

[0100] Apply an adaptive median filter:

[0101] I denoised (i,j)=median({I blur (i+k,j+l)|-W adaptive / 2≤k,l≤W adaptive / 2});

[0102] In the formula, I denoised (i, j) is the value of pixel (i, j) after median filtering; I blur (i+k,j+l) is the pixel value of the medical image at position (i+k,j+l) after Gaussian filtering; W adaptive The window size of the median filter after adaptive adjustment.

[0103] The adaptive Gaussian-median filter in this embodiment can adaptively adjust the filtering parameters, combines the smoothing effect of the adaptive Gaussian filter and the edge preservation ability of the median filter, strikes a balance between computational efficiency and implementation difficulty, and is particularly suitable for medical image preprocessing.

[0104] Adaptive adjustment of filter parameters based on local image characteristics (such as mean and variance) enables the filter to perform fine processing in areas with different noise levels and detail complexity. In areas with high noise, the smoothing effect is automatically increased, and in areas with rich details, smoothing is reduced to retain more details. Different areas in medical images may have different noise levels and detail complexity. Adaptive filter parameter adjustment can effectively cope with this variability and improve overall image quality. For example, in important areas such as tumor edges, adaptive adjustment can better protect the details of these areas and improve the accuracy of diagnosis.

[0105] Medical images may contain many types of noise. Gaussian filters perform well in smoothing Gaussian noise (such as electronic noise), while median filters are effective in removing impulse noise (such as salt and pepper noise). The combination of the two can more comprehensively remove various noises and improve image quality. At the same time, edges and details are crucial to medical diagnosis, especially when identifying lesions, tissue boundaries, and organ contours. The edge retention ability of the median filter ensures that this important information will not be blurred.

[0106] In clinical environments, real-time or near-real-time image processing is very important. The adaptive Gaussian-median filter can meet the requirements of real-time processing while ensuring the denoising effect. Compared with more complex filtering methods (such as adaptive bilateral filters), the adaptive Gaussian-median filter has moderate computational complexity, is easier to implement and optimize, and is suitable for real-time or near-real-time processing needs. It is also suitable for various medical image types such as CT, MRI, ultrasound, and X-ray, and has high robustness and stability.

[0107] The following are some examples:

[0108] For CT scan images: Gaussian noise and impulse noise are common in CT images. Adaptive Gaussian-median filter can effectively remove these two types of noise while retaining important anatomical structures such as blood vessels and tissue boundaries;

[0109] For MRI images: MRI images are very detailed and have varying noise levels. The adaptive Gaussian-median filter enhances the smoothing effect in noisy areas and retains more details in detailed areas by adaptively adjusting the filtering parameters, which helps to clearly display the lesion site;

[0110] For ultrasound images: Speckle noise is common in ultrasound images. The adaptive Gaussian-median filter can reduce the impact of speckle noise on image quality while retaining the boundary information of tissues and organs through the edge preservation characteristics of the median filter.

[0111] The image enhancement unit is used to integrate the contrast-limited adaptive histogram equalization (CLAHE) and multi-scale Retinex algorithm to enhance the contrast and details of medical image data, making the lesion area more obvious, which is helpful for subsequent segmentation and feature extraction;

[0112] CLAHE is an improved form of histogram equalization, which can perform histogram equalization in local areas to avoid the problem of overall contrast distortion of the image. In medical image processing, CLAHE is often used to enhance the details in the image, especially in images with a wide range of grayscale changes, such as X-rays, CT scans, and MRI images. The Retinex algorithm is a classic method for image enhancement, which simulates the human eye's perception mechanism of light and surface reflection of objects; the multi-scale Retinex algorithm combines image decomposition and Retinex enhancement processes at multiple scales, which can better retain the detailed information of the image and suppress the impact of light changes on the image. In medical image processing, the multi-scale Retinex algorithm is used to enhance the contrast and details of the image, making it easier for doctors to observe the lesion area.

[0113] The image standardization unit is used to standardize medical image data across devices and modalities. Through gamma correction and intensity normalization, medical image data from different sources have consistent brightness and contrast characteristics, which can improve data consistency between different medical imaging devices and facilitate unified analysis.

[0114] Gamma correction is used to adjust the brightness and contrast of an image. By performing nonlinear transformation on the grayscale of an image, it can better adapt to images under different imaging devices and lighting conditions. Intensity normalization maps the grayscale value range of an image to a uniform interval to ensure that the brightness and contrast between different images are consistent. In medical image processing, gamma correction and intensity normalization can be used to make medical image data from different sources have consistent visual characteristics, thereby facilitating doctors to conduct unified analysis and diagnosis.

[0115] Image segmentation module: used to segment the lesion area and organ area from the preprocessed medical image data through a fusion model. The fusion model is a fusion model of multi-scale U-Net, PSPNet (pyramid scene parsing network), hybrid CNN-Transformer and multi-task learning;

[0116] U-Net is a classic image segmentation network that uses a symmetrical encoder-decoder structure. The encoder part gradually reduces the spatial resolution and improves the feature representation capability; the decoder part gradually restores the spatial resolution and combines the feature maps of the corresponding layer of the encoder to achieve fine segmentation. Multi-scale U-Net uses a multi-scale feature extraction mechanism based on U-Net, and captures image features of different scales by introducing multiple downsampling and upsampling operations of different scales.

[0117] PSPNet (Pyramid Scene Parsing Network), translated into Chinese as Pyramid Scene Parsing Network, is mainly used for image segmentation. This architecture introduces a pyramid pooling module to capture contextual information at different scales. Pyramid Pooling can extract global and local contextual information at different scales, which helps to better understand the semantic content in the image and thus improve segmentation performance.

[0118] The hybrid CNN-Transformer combines the advantages of CNN and Transformer in the encoding stage. It extracts local features through CNN and captures global and long-distance dependency features through Transformer. It can capture high-level local features while maintaining long-distance dependencies from different input patches. It adopts a dual decoder structure, with one decoder focusing on the segmentation of lesion areas and the other decoder focusing on the segmentation of organ areas. In addition, an attention gate is introduced in each decoder to further improve the segmentation performance of the model by selectively focusing on important feature areas. The dual decoder design uses attention gates, residual blocks, and upsampling layers to more effectively segment lesions and organ areas. In particular, in the case of some lesions affecting multiple organs, multi-task learning is achieved to simultaneously process the segmentation tasks of lesions and organs, thereby utilizing the complementary information of the two tasks and improving the accuracy and robustness of the overall segmentation.

[0119] Multi-task learning (MTL) is a machine learning method that aims to improve the generalization ability of the model by learning multiple related tasks at the same time. MTL can use the correlation between tasks to improve the learning effect of each task by sharing information and features between tasks. In medical image processing, it is usually necessary to segment multiple different types of targets at the same time, such as tumors, organs, lesions, etc. The MTL model can process these tasks simultaneously under the same framework and improve segmentation accuracy by sharing underlying features.

[0120] Specifically, the encoder part of the fusion model combines multi-scale U-Net and hybrid CNN-Transformer, taking advantage of the multi-resolution feature extraction capability of multi-scale U-Net and the global feature capture capability of hybrid CNN-Transformer; in the encoder part of U-Net, the architecture of multi-scale U-Net is used to extract multi-resolution features through convolutional layers of different scales, and perform multi-scale feature extraction;

[0121] In each encoder layer, a multi-scale convolution layer is used to extract features of different resolutions. The formula is:

[0122] x b =Conv a×a (x b -1);

[0123] In the formula, x b is the feature map of layer b, Conv a×a Represents the convolution operation of a×a;

[0124] Based on multi-scale feature extraction, a Transformer module is inserted after each U-Net encoder layer to capture global features through the self-attention mechanism, including:

[0125] Divide the input image into non-overlapping patches x v :

[0126]

[0127] Where S is the size of each patch, N is the total number of patches, X is the number of channels, and R represents the real space (usually used to represent the dimensions of matrices or vectors).

[0128] Each patch is transformed by a linear matrix Mapping to the embedding space, we get the embedding vector z0:

[0129] z0=[x v1 E,x v2 E,...,x vN E];

[0130] Where D is the dimension of the embedding space; x v1 Indicates the first patch, x v2 Indicates the second patch, x vN Indicates the Nth patch;

[0131] Multi-head self-attention (MSA) is applied in the Transformer layer to capture global dependencies:

[0132] z′ g =MSA(LN(z g -1))+z g -1;

[0133] In the formula, z g is the embedding vector of the Transformer encoder at layer g; LN represents LayerNorm (layer normalization), MSA represents the multi-head self-attention mechanism; z′ g Represents the intermediate representation after multi-head self-attention mechanism (MSA) and LayerNorm processing.

[0134] The decoder part of the fusion model adopts a dual decoder structure of hybrid CNN-Transformer, which serves as a lesion decoder and an organ decoder respectively. The lesion decoder focuses on the segmentation of the lesion area, adopts a decoder architecture based on a multi-scale U-Net, and restores the spatial resolution through layer-by-layer upsampling;

[0135] The lesion decoder uses convolutional layers and upsampling layers to restore the spatial resolution layer by layer, as follows:

[0136] d g1 =ResB(Concatenate[AttGate(x g ,UpSample(x g+1)),UpSample(x g+1 )]);

[0137] Where, d g1 represents the output feature map of the lesion decoder at layer g; x g represents the input feature map of the decoder at layer g, x g+1 Represents the input feature map of the decoder at the g+1 layer; ResB represents the residual block; AttGate represents the attention gate; UpSample represents upsampling; Concatenate represents the concatenation operation;

[0138] The organ decoder focuses on the segmentation of organ regions and uses the pyramid pooling module of PSPNet to capture multi-scale contextual information to further improve the accuracy of organ segmentation.

[0139] The organ decoder uses the pyramid pooling module of PSPNet to capture multi-scale contextual information, the formula is:

[0140] d g2 =PSP(ResB(Concatenate[AttGate(x g ,UpSample(x g+1 )),UpSample(x g+1 )]));

[0141] Where, d g2 represents the output feature map of the organ decoder at the gth layer; PSP represents the pyramid pooling module in PSPNet, which is used to capture multi-scale contextual information;

[0142] The fusion model uses a multi-task learning framework. The encoder part fuses multi-scale U-Net and hybrid CNN-Transformer as a shared encoder to extract common features of the input image; the lesion decoder and organ decoder are independent decoders, which handle the segmentation tasks of lesion areas and organ areas respectively; during the training process, the lesion segmentation and organ segmentation tasks are optimized simultaneously through a multi-task loss function (such as weighted cross entropy loss), the weights are adjusted according to the importance of the tasks, the contributions of different tasks are balanced, the segmentation performance is further improved, and the shared features are used to improve the generalization ability and robustness of the model.

[0143] Use the multi-task loss function L hybrid , combined with cross entropy loss (CE) and Dice loss for optimization:

[0144]

[0145] Where, L LES-CE and L LES-Diceare the CE loss and Dice loss of the diseased branch, L O-CE is the CE loss of the organ branch, ω, γ is the weight parameter.

[0146] For simple classification tasks, it may be sufficient to directly use the segmented lesion area and / or organ area images for decision analysis. For complex classification tasks, especially those involving small differences or high-dimensional features, further feature extraction may be required. Whether dealing with different organs, lesion types or different imaging modalities, in-depth feature extraction helps to improve the generalization ability and robustness of the system. Therefore, in the embodiment of the present invention, feature extraction can be performed again on the segmented lesion area image and / or organ area image.

[0147] Feature extraction module: used to extract key feature information again from the segmented lesion area image and / or organ area image. Common methods include: deep feature extraction based on convolutional neural network (CNN); feature extraction using pre-trained models (such as ResNet, VGG); traditional image processing methods (such as morphological features, texture features) combined with deep learning methods.

[0148] The key feature information extracted may include:

[0149] Shape characteristics of the lesion area: including the shape, outline, boundary and other information of the lesion, which can help identify the type and severity of the lesion;

[0150] Size features: including measurements such as the area and volume of the lesion area, which help assess the extent of the lesion;

[0151] Position features: including the specific position and distribution of the lesion area in the image. These features can assist in locating the lesion;

[0152] Texture features:

[0153] Gray Level Co-occurrence Matrix (GLCM): used to extract texture information of images, including contrast, correlation, entropy and other indicators. These texture features can help distinguish different types of tissues and lesions;

[0154] Local Binary Pattern (LBP): used to describe the local texture features of an image, especially suitable for detecting subtle texture changes;

[0155] Density and Strength Characteristics:

[0156] Gray value histogram: used to analyze the distribution of different gray levels in the image, which can reveal the density and intensity characteristics of the tissue;

[0157] Mean and standard deviation: used to describe the average gray value and gray value variation of different areas in the image. These indicators can reflect the uniformity and variability of the tissue;

[0158] Morphological characteristics:

[0159] Edge detection: Use various edge detection algorithms (such as Canny and Sobel) to extract edge information in the image, which is very important for identifying the boundaries and morphological features of lesions;

[0160] Morphological operations: such as dilation, erosion, opening and closing operations, are used to process the morphological features of images and help extract and enhance the morphological information of the target area.

[0161] Decision analysis module: Based on the extracted key feature information, PU semi-supervised learning is used to combine positive examples and unlabeled data to train the classification model, and the semi-supervised disease classification strategy is implemented through the classification model;

[0162] Annotation of medical images requires the expertise of well-trained clinical professionals. However, these experts often face time constraints due to their busy schedules, which makes it difficult for them to manually review a large number of image samples. Therefore, the lack of labeled images hinders the development of effective classifiers. Semi-supervised learning is the process of automatically utilizing unlabeled samples to improve learning performance without relying on external interactions.

[0163] PU semi-supervised learning (Positive-Unlabeled Learning, PU Learning) focuses on solving classification problems in data sets with only positive samples and unlabeled samples. Unlike traditional semi-supervised learning, PU semi-supervised learning assumes that negative samples do not exist or are very few, and mainly uses unlabeled data to mine potential negative sample information. The basic process mainly includes:

[0164] Initialize the model: Use existing positive samples and some unlabeled samples to train the initial classifier, usually through traditional supervised learning methods;

[0165] Negative sample generation: Use the initial classifier to predict the unlabeled samples and add the samples predicted as negative to the negative sample set;

[0166] Model update: Retrain the classifier using positive samples and generated negative samples, and iterate the above steps until the model converges or reaches the predetermined performance indicator.

[0167] In a specific embodiment, the method of using PU semi-supervised learning to train a classification model in combination with positive examples and unlabeled data includes:

[0168] The PU dataset is constructed by extracting key feature information, and h data are randomly selected and manually labeled as positive samples, and the rest are unlabeled data;

[0169] Construct a classification model based on generative adversarial networks, use a classifier instead of a generator to determine the probability that any data in the unlabeled data is a positive sample, and pass the data determined as a positive sample to the discriminator, and use the discriminator to determine the authenticity of the data as a positive sample;

[0170] The objective function of the classification model is constructed based on the Holder divergence. The objective function f(D, C) is expressed as:

[0171]

[0172] Where C is the classifier, D is the discriminator; M is the number of samples in the PU dataset, M0 is the number of unlabeled data in the PU dataset; P m is the probability distribution of positive samples in the PU dataset, D m is the discriminant distribution of the mth data in the PU dataset; H is the Holder divergence; δ is a balance hyperparameter used to adjust the weights between different loss terms; is the discriminant distribution of the mth data in the unlabeled data, is the probability distribution of the classifier for the mth data in the unlabeled data; Represents the inverse distribution of the classifier, which is the supplementary distribution of the classifier for the mth data in the unlabeled data.

[0173] Result output module: The classification results of the classification model are displayed through a graphical user interface and stored in a database or medical health record system.

[0174] In summary, the present invention uses multi-scale convolutional layers to extract features of different resolutions in each layer of encoder, which can more comprehensively capture various details and global information in the image, thereby improving the representation ability and analysis accuracy of image features; inserting a Transformer module after each U-Net encoder layer, using a multi-head self-attention mechanism to capture global dependencies, enhancing the model's understanding of the relationship between distant pixels in the image, and further improving the model's expressiveness and accuracy; the lesion decoder uses convolutional layers and upsampling layers to restore spatial resolution layer by layer, ensuring that the details of the original image are retained as much as possible during the decoding process, and improving the accuracy of the segmentation result; the organ decoder uses the pyramid pooling module of PSPNet to capture multi-scale contextual information, helping the model to better understand the image content at different scales and improve the accuracy of organ segmentation. Using a multi-task loss function combined with cross entropy loss and Dice loss for optimization, the lesion branch and the organ branch can be optimized at the same time, so that the model can achieve better performance when processing different tasks, and improve the overall analysis effect. Semi-supervised learning is used to effectively utilize limited labeled data and a large amount of unlabeled data, improve the ability to extract image features and the accuracy of classification and segmentation, and significantly improve the performance and accuracy of the image analysis and processing system.

[0175] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or other arbitrary combinations. When implemented using software, it can be implemented in whole or in part in the form of a computer program product, which includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0176] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0177] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A medical image analysis and processing system based on image analysis, characterized in that: The system comprises: An image acquisition module is connected to the interface of the medical imaging device and is used to automatically receive or manually upload the original medical image data; a data preprocessing module is used to preprocess the collected medical image data through image processing technology; An image segmentation module, used to segment lesion area images and organ area images from preprocessed medical image data through a fusion model, wherein the fusion model is specifically a fusion model of multi-scale U-Net, PSPNet, hybrid CNN-Transformer and multi-task learning; A feature extraction module is used to extract key feature information again from the segmented lesion region image and / or organ region image, wherein the key feature information includes shape feature, size feature, position feature, texture feature, density and intensity feature of the lesion region; The decision analysis module uses PU semi-supervised learning to train the classification model based on the extracted key feature information and combines positive samples and unlabeled data, and implements the semi-supervised disease classification strategy through the classification model; A result output module, used to display the classification results of the classification model through a graphical user interface and store them in a database or a medical health record system; The hybrid CNN-Transformer extracts local features through CNN and captures global and long-distance dependency features through Transformer. It adopts a dual decoder structure, one decoder is used for lesion area segmentation, and the other decoder is used for organ area segmentation. An attention gate is introduced in each decoder to simultaneously process lesion and organ segmentation tasks. The fusion model specifically includes: The encoder part combines multi-scale U-Net and hybrid CNN-Transformer. In the encoder part of U-Net, the architecture of multi-scale U-Net is used to extract multi-resolution features through convolutional layers of different scales to perform multi-scale feature extraction; The decoder part adopts a hybrid CNN-Transformer dual decoder structure, which serves as a lesion decoder and an organ decoder respectively. The lesion decoder is used for the segmentation of lesion areas, and adopts a decoder architecture based on a multi-scale U-Net to restore spatial resolution through layer-by-layer upsampling. The organ decoder is used for the segmentation of organ areas, and uses the pyramid pooling module of PSPNet to capture multi-scale contextual information. Using a multi-task learning framework, the encoder part fuses multi-scale U-Net and hybrid CNN-Transformer as a shared encoder to extract common features of the input image; the lesion decoder and organ decoder are independent decoders to handle the segmentation tasks of the lesion area and organ area respectively; during the training process, the multi-task loss function is used to optimize the segmentation tasks of the lesion area and organ area at the same time, and the weights are adjusted according to the importance of the tasks; In the encoder part of the fusion model, a multi-scale convolution layer is used in each encoder layer to extract features of different resolutions. The formula is: x b =Conv a×a (x b -1); In the formula, x b is the feature map of layer b, Conv a×a Represents the convolution operation of a×a; Based on multi-scale feature extraction, a Transformer module is inserted after each U-Net encoder layer, and the Transformer layer is used to capture global features, including: Divide the input image into non-overlapping patches x v : Where S is the size of each patch, N is the total number of patches, X is the number of channels, and R represents the real number space. Each patch is transformed by a linear matrix Mapping to the embedding space, we get the embedding vector z0: z0=[x v1 E,x v2 E,...,x vN E]; Where Y is the dimension of the embedding space; x v1 Indicates the first patch, x v2 Indicates the second patch, x vN represents the Nth patch; a multi-head self-attention mechanism is applied in the Transformer layer to capture global dependencies: z′ g =MSA(LN(z g -1))+z g -1; In the formula, z g is the embedding vector of the Transformer encoder at layer g; LN represents layer normalization, MSA represents the multi-head self-attention mechanism; z′ g Represents the intermediate representation after multi-head self-attention mechanism and layer normalization; The lesion decoder uses convolutional layers and upsampling layers to restore the spatial resolution layer by layer, as follows: d g1 =ResB(Concatenate[AttGate(x g ,UpSample(x g+1 )),UpSample(x g+1 )]); Where, d g1 represents the output feature map of the lesion decoder at layer g; x g represents the input feature map of the decoder at layer g, x g+1 Represents the input feature map of the decoder at the g+1 layer; ResB represents the residual block; AttGate represents the attention gate; UpSample represents upsampling; Concatenate represents the concatenation operation; The organ decoder uses the pyramid pooling module of PSPNet to capture multi-scale contextual information, the formula is: d g2 =PSP(ResB(Concatenate[AttGate(x g ,UpSample(x g+1 )),UpSample(x g+1 )])); Where, d g2 represents the output feature map of the organ decoder at the gth layer; PSP represents the pyramid pooling module in PSPNet.

2. A medical image analysis and processing system based on image analysis according to claim 1, characterized in that: The image acquisition module also includes an intelligent image quality detection unit and an expansion unit; The intelligent image quality detection unit is used to integrate an image quality detection algorithm to automatically detect and filter low-quality images that are blurred or abnormally exposed; The expansion unit is used to automatically adapt to the interface protocols of different medical imaging devices, and realize image acquisition of medical imaging devices of various models through a universal interface.

3. A medical image analysis and processing system based on image analysis according to claim 1, characterized in that: The data preprocessing module includes an adaptive filtering unit, an image enhancement unit and an image normalization unit; The adaptive filtering unit is used to perform denoising on the medical image data by adaptively adjusting the filtering parameters according to the local characteristics of the medical image data through an adaptive Gaussian-median filter; The image enhancement unit is used to integrate contrast-limited adaptive histogram equalization and multi-scale Retinex algorithm to enhance the contrast of medical image data; The image standardization unit is used to perform standardization processing on cross-device and cross-modality medical image data, and through gamma correction and intensity normalization, the medical image data from different sources have consistent brightness and contrast characteristics.

4. A medical image analysis and processing system based on image analysis according to claim 3, characterized in that: The adaptive Gaussian-median filter specifically includes: Calculate the mean and variance of the local area of ​​the medical image, and adaptively adjust the filter parameters according to the local image characteristics. The formula is: In the formula, μ i,j is the mean of the local area around pixel (i, j), k is the local window radius, and the window size is (2k+1)×(2k+1); u represents the offset of the pixel in the window relative to the center pixel i in the row direction, and v represents the offset of the pixel in the window relative to the center pixel j in the column direction; I(i+u,j+v) is the pixel value of medical image I at position (i+u,j+v); σ 2 i,j is the variance of the local area around pixel (i, j); Dynamically adjust the standard deviation of the Gaussian filter based on local statistical characteristics: s adaptive (i,j)=a·s i,j +b; In the formula, σ adaptive (i, j) represents the standard deviation of the adaptively adjusted Gaussian filter for pixel (i, j); α is the scaling factor; σ i,j is the standard deviation of the local area around pixel (i, j); β is the offset; Apply an adaptive Gaussian filter for convolution: In the formula, G adaptive (x,y;σ adaptive ) is the adaptive Gaussian kernel, x and y are the position coordinates relative to the center of the filter, σ adaptive is the standard deviation of the Gaussian filter after adaptive adjustment; I blur (i, j) is the value of pixel (i, j) after Gaussian filtering; I(i+k, j+l) represents the pixel value of the medical image at position (i+k, j+l); k and l are the row and column offsets, respectively; K and L are the radii of the Gaussian kernel; G adaptive (k,l;σ adaptive (i,j)) is the adaptive Gaussian kernel value; According to the local statistical characteristics, the window size of the median filter is dynamically adjusted: Where W adaptive (i, j) is the adaptively adjusted median filter window size for pixel (i, j); W max , W min are the maximum window size and minimum window size of the median filter respectively; σ max is the maximum value of the standard deviation of all local regions in the medical image; Apply an adaptive median filter: I denoised (i,j)=median({I blur (i+k,j+l)∣-W adaptive / 2≤k,l≤W adaptive / 2}); In the formula, I denoised (i, j) is the value of pixel (i, j) after median filtering; I blur (i+k,j+l) is the pixel value of the medical image at position (i+k,j+l) after Gaussian filtering; W adaptive The window size of the median filter after adaptive adjustment.

5. A medical image analysis and processing system based on image analysis according to claim 1, characterized in that: In the multi-task learning framework, the multi-task loss function L is used. hybrid , combined with cross entropy loss and Dice loss for optimization: Where, L LES-CE and L LES-Dice are the cross entropy loss and Dice loss of the diseased branch, L O-CE is the CE loss of the organ branch, ω, γ is the weight parameter.

6. A medical image analysis and processing system based on image analysis according to claim 1, characterized in that: The method of using PU semi-supervised learning to train the classification model in combination with positive samples and unlabeled data specifically includes: The PU dataset is constructed by extracting key feature information, and h data are randomly selected and manually labeled as positive samples, and the rest are unlabeled data; Construct a classification model based on generative adversarial networks, use a classifier instead of a generator to determine the probability that any data in the unlabeled data is a positive sample, and pass the data determined as a positive sample to the discriminator, and use the discriminator to determine the authenticity of the data as a positive sample; The objective function of the classification model is constructed based on the Holder divergence, and the objective function f(D, C) is expressed as: Where C is the classifier, D is the discriminator; M is the number of samples in the PU dataset, M0 is the number of unlabeled data in the PU dataset; P m is the probability distribution of positive samples in the PU dataset, D m is the discriminant distribution of the mth data in the PU dataset; H is the Holder divergence; δ is a balance hyperparameter used to adjust the weights between different loss terms; is the discriminant distribution of the mth data in the unlabeled data, is the probability distribution of the classifier for the mth data in the unlabeled data; Represents the inverse distribution of the classifier, which is the supplementary distribution of the classifier for the mth data in the unlabeled data.

Images