Image recognition and disease prediction model and system based on low-density lipoprotein electrophoresis

Abstract

The invention discloses an image recognition and disease prediction model and method based on low-density lipoprotein electrophoresis, and relates to the field of image recognition. Comprising the steps of collecting multiple groups of disease positive samples for training to obtain a disease prediction training model; testing a health negative sample of the patient through the disease prediction training model; when the output identification indicates that each disease type is suffered, recursively removing each key feature of the healthy negative sample until the output identification of the disease prediction training model indicates that each disease type is not suffered, collecting each removed key feature as an unimportant feature, and finally obtaining an unimportant feature set of each disease type; collecting multiple groups of disease prediction optimization data; each group of disease prediction optimization data comprises a health negative sample of the patient, key features obtained after unimportant features are recursively removed each time, and an identifier used for representing that the patient does not suffer from various disease types; the disease prediction training model is trained by multiple groups of disease prediction optimization data, so that the speed and accuracy of low-density lipoprotein electrophoresis image detection are improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a model and method for low-density lipoprotein electrophoresis image recognition and disease prediction. Background Art

[0002] Traditional methods of low-density lipoprotein (LDL) typing detection technology, such as ultracentrifugation, gradient gel electrophoresis and high-performance liquid chromatography, have some limitations in lipoprotein subtype analysis, including complex operation, long detection cycle, insufficient resolution and low specificity, which limit their wide application. Usually, the relevant experimental data for low-density lipoprotein typing detection is obtained through image recognition technology or detection equipment, and the disease prediction results are obtained by manually observing pathological images, but such manual prediction results are inefficient and prone to errors. Therefore, how to design a model and system based on low-density lipoprotein electrophoresis image recognition and disease prediction, realize low-density lipoprotein electrophoresis image recognition and disease prediction, and improve the speed and accuracy of disease prediction is the main research difficulty. Summary of the invention

[0003] The technical problem to be solved by the present invention is how to realize low-density lipoprotein electrophoresis image recognition and disease prediction, and improve the speed and accuracy of disease prediction. The purpose is to provide a low-density lipoprotein electrophoresis image recognition and disease prediction model and system to solve the above technical problems.

[0004] The present invention is achieved through the following technical solutions:

[0005] Based on low-density lipoprotein electrophoresis image recognition and disease prediction model, including:

[0006] Collect diseased positive samples and healthy negative samples of low-density lipoprotein electrophoresis images;

[0007] Extract key features of preprocessed low-density lipoprotein electrophoresis images;

[0008] Collecting multiple sets of disease prediction training data; each set of disease prediction training data includes a positive sample of a patient, at least one key feature, one or more disease types, and an identifier for indicating whether the patient suffers from each disease type; and obtaining a disease prediction training model by training the multiple sets of disease prediction training data;

[0009] Collecting multiple sets of disease prediction verification data; each set of disease prediction verification data includes a healthy negative sample of the patient, at least one key feature, and one or more disease types; the multiple sets of disease prediction verification data are tested by the disease prediction training model respectively;

[0010] When the output is identified as suffering from each disease type, the key features of the healthy negative samples are recursively removed until the disease prediction training model outputs an identification of not suffering from each disease type, and the removed key features are collected as unimportant features; finally, an unimportant feature set of each disease type is obtained;

[0011] Collect multiple sets of disease prediction optimization data; each set of the disease prediction optimization data includes healthy negative samples of patients, key features after recursively removing at least one unimportant feature each time, one or more disease types, and identifiers used to indicate that the patient does not suffer from each disease type; train the disease prediction training model using the multiple sets of disease prediction optimization data.

[0012] The basis for recursively removing each key feature is to preferentially delete the same features between the healthy negative sample and the diseased positive sample of the same disease type, and the same features between the healthy negative samples of the same disease type.

[0013] The key features of the pre-processed low-density lipoprotein electrophoresis image are extracted using a pre-trained feature extraction model; the key features include any one or more of color, texture, shape, size, position, and context.

[0014] The preprocessing of the low-density lipoprotein electrophoresis image includes any one or more operations of denoising, image enhancement and image normalization.

[0015] After extracting the key features of the pre-processed low-density lipoprotein electrophoresis image, the key features are pre-processed; pre-processing the key features includes standardization and normalization operations.

[0016] The key features are recursively removed, and features are selected using a recursive feature elimination method.

[0017] When the plurality of groups of the disease prediction training data are trained to obtain a disease prediction training model, the disease prediction training model is trained through the following steps:

[0018] Build deep learning models; compile and train deep learning models; train random forests and gradient boosting machines as base learners and use the predictions of deep learning models as additional features;

[0019] The prediction results of the random forest, gradient boosting machine and deep learning model are combined using StackingClassifier to form an integrated learning model as the disease prediction training model.

[0020] Based on low-density lipoprotein electrophoresis image recognition and disease prediction system, including:

[0021] Image sample collection module: collects diseased positive samples and healthy negative samples of low-density lipoprotein electrophoresis images;

[0022] Image feature extraction module: extract key features of the pre-processed low-density lipoprotein electrophoresis image;

[0023] Disease prediction training module: collect multiple sets of disease prediction training data; each set of disease prediction training data includes a positive sample of a disease, at least one key feature, one or more disease types, and an identifier for indicating whether the patient suffers from each disease type; a disease prediction training model is obtained by training multiple sets of the disease prediction training data;

[0024] Disease prediction test module: collect multiple sets of disease prediction verification data; each set of disease prediction verification data includes a healthy negative sample of the patient, at least one key feature, and one or more disease types; the multiple sets of disease prediction verification data are tested by the disease prediction training model respectively;

[0025] Disease feature identification module: when the output is identified as suffering from each disease type, recursively remove the key features of the healthy negative sample until the disease prediction training model outputs an identification of not suffering from each disease type, and collects the removed key features as unimportant features; finally, an unimportant feature set of each disease type is obtained;

[0026] Disease prediction optimization module: collect multiple groups of disease prediction optimization data; each group of the disease prediction optimization data includes healthy negative samples of patients, key features after recursively removing at least one unimportant feature each time, one or more disease types, and identifiers used to indicate that the patient does not suffer from each disease type; the disease prediction training model is trained using multiple groups of the disease prediction optimization data.

[0027] An electronic device comprises a memory, a processor and a computer program running on the processor, wherein the processor implements the low-density lipoprotein electrophoresis image recognition and disease prediction model when executing the computer program.

[0028] A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the low-density lipoprotein electrophoresis image recognition and disease prediction model as described in any one of claims 1 to 7.

[0029] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0030] Collect multiple groups of disease prediction training data; each group of disease prediction training data includes diseased positive samples, key features extracted from samples, one or more disease types, and identifiers used to indicate whether the patient suffers from each disease type; obtain a disease prediction training model through training; use diseased healthy negative samples to test the disease prediction training model, select unimportant feature sets for each disease type based on the test results, and train the disease prediction training model with optimized healthy negative samples. The present invention can identify unimportant features of various disease types based on low-density lipoprotein electrophoresis images, and implement disease prediction for the model through healthy negative samples of low-density lipoprotein electrophoresis images, thereby realizing an automated process based on low-density lipoprotein electrophoresis image recognition and disease prediction, and improving the speed and accuracy of disease prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative work. In the drawings:

[0032] Figure 1 This is a flow chart of the low-density lipoprotein electrophoresis image recognition and disease prediction model and method according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0034] Example

[0035] like Figure 1 As shown, the embodiment of the present application provides a low-density lipoprotein electrophoresis image recognition and disease prediction model, including:

[0036] Collect diseased positive samples and healthy negative samples of low-density lipoprotein electrophoresis images;

[0037] Extract key features of preprocessed low-density lipoprotein electrophoresis images;

[0038] Collecting multiple sets of disease prediction training data; each set of disease prediction training data includes a positive sample of a patient, at least one key feature, one or more disease types, and an identifier for indicating whether the patient suffers from each disease type; and obtaining a disease prediction training model by training the multiple sets of disease prediction training data;

[0039] Collecting multiple sets of disease prediction verification data; each set of disease prediction verification data includes a healthy negative sample of the patient, at least one key feature, and one or more disease types; the multiple sets of disease prediction verification data are tested by the disease prediction training model respectively;

[0040] When the output is identified as suffering from each disease type, the key features of the healthy negative samples are recursively removed until the disease prediction training model outputs an identification of not suffering from each disease type, and the removed key features are collected as unimportant features; finally, an unimportant feature set of each disease type is obtained;

[0041] Collect multiple sets of disease prediction optimization data; each set of the disease prediction optimization data includes healthy negative samples of patients, key features after recursively removing at least one unimportant feature each time, one or more disease types, and identifiers used to indicate that the patient does not suffer from each disease type; train the disease prediction training model using the multiple sets of disease prediction optimization data.

[0042] The basis for recursively removing each key feature is to preferentially delete the same features between the healthy negative sample and the diseased positive sample of the same disease type, and the same features between the healthy negative samples of the same disease type.

[0043] The key features of the pre-processed low-density lipoprotein electrophoresis image are extracted using a pre-trained feature extraction model; the key features include any one or more of color, texture, shape, size, position, and context.

[0044] The preprocessing of the low-density lipoprotein electrophoresis image includes any one or more operations of denoising, image enhancement and image normalization.

[0045] After extracting the key features of the pre-processed low-density lipoprotein electrophoresis image, the key features are pre-processed; pre-processing the key features includes standardization and normalization operations.

[0046] The key features are recursively removed, and features are selected using a recursive feature elimination method.

[0047] When the plurality of groups of the disease prediction training data are trained to obtain a disease prediction training model, the disease prediction training model is trained through the following steps:

[0048] Build deep learning models; compile and train deep learning models; train random forests and gradient boosting machines as base learners and use the predictions of deep learning models as additional features;

[0049] The prediction results of the random forest, gradient boosting machine and deep learning model are combined using StackingClassifier to form an integrated learning model as the disease prediction training model.

[0050] Optionally, when collecting diseased positive samples and healthy negative samples of low-density lipoprotein electrophoresis images, a high-resolution electrophoresis instrument is used to implement the electrophoresis experiment, which can ensure the clarity and accuracy of the image. Optionally, the image acquisition conditions include: the resolution uses at least 2048x2048 pixels to clearly capture the characteristics of LDL particles; the illumination conditions use uniform illumination to reduce image noise caused by uneven illumination; the image format is JPEG or PNG format to ensure that the image quality is not damaged by compression. After the electrophoresis is completed, the LDL sample after electrophoresis is photographed by a high-resolution camera, and the collected image will be used for subsequent preprocessing and analysis.

[0051] Preprocessing of low-density lipoprotein electrophoresis images. When denoising, use the denoising autoencoder model to restore clear images from noisy LDL electrophoresis images. Use the noisy image as input and the original clean image as the target output to train the denoising autoencoder. Optimize the weights of the model by minimizing the reconstruction error so that the model can learn how to effectively remove noise. Mixed precision training can be introduced using Float16 precision to speed up the training process, reduce memory usage, and improve training efficiency. Then use the trained denoising autoencoder to perform denoising predictions on the noisy images of the test set to verify the performance and denoising ability of the model on actual noisy images.

[0052] The code for the denoising autoencoder is as follows:

[0053] # Enable mixed precision training

[0054] policy=mixed_precision.Policy('mixed_float16')

[0055] mixed_precision.set_policy(policy)

[0056] def build_denoising_autoencoder(input_shape):

[0057] inputs=Input(shape=input_shape)

[0058] #Encoder

[0059] x = Conv2D(32, (3, 3), padding='same')(inputs)

[0060] x = LeakyReLU(0.2)(x)

[0061] x = MaxPooling2D((2, 2), padding='same')(x)

[0062] x = Conv2D(64, (3, 3), padding='same')(x)

[0063] x = BatchNormalization()(x)

[0064] x = LeakyReLU(0.2)(x)

[0065] x = MaxPooling2D((2, 2), padding='same')(x)

[0066] # Bottleneck layer

[0067] x = Conv2D(128, (3, 3), padding='same')(x)

[0068] x = BatchNormalization()(x)

[0069] x = LeakyReLU(0.2)(x)

[0070] # Decoder

[0071] x = Conv2D(64, (3, 3), padding='same')(x)

[0072] x = BatchNormalization()(x)

[0073] x = LeakyReLU(0.2)(x)

[0074] x = UpSampling2D((2, 2))(x)

[0075] x = Conv2D(32, (3, 3), padding='same')(x)

[0076] x = BatchNormalization()(x)

[0077] x = LeakyReLU(0.2)(x)

[0078] x = UpSampling2D((2,2))(x)

[0079] x=Conv2D(1,(3,3),padding='same')(x)

[0080] outputs=BatchNormalization()(x)

[0081] model=Model(inputs=inputs,outputs=outputs)

[0082] model.compile(optimizer=Adam(learning_rate=0.001),loss='mean_squared_error')

[0083] return model

[0084] input_shape = (2048, 2048, 1) # Image size is 2048x2048

[0085] autoencoder=build_denoising_autoencoder(input_shape)

[0086] #Define callback function

[0087] reduce_lr=ReduceLROnPlateau(monitor='loss', factor=0.2, patience=5, min_lr=0.0001)

[0088] early_stopping=EarlyStopping(monitor='loss',patience=10,restore_best_weights=True)

[0089] #Train Autoencoder, original_images and noisy_images are original and noisy image data

[0090] autoencoder.fit(noisy_images, original_images, epochs=55, batch_size=120, shuffle=True, validation_split=0.2, callbacks=[reduce_lr, early_stopping])

[0091] # Denoising prediction

[0092] denoised_images=autoencoder.predict(noisy_images_test)

[0093] The preprocessing of low-density lipoprotein electrophoresis images can be done through data enhancement, such as enhancing the image dataset by adding multiple noises (such as Gaussian noise, salt and pepper noise, speckle noise, stripe noise and quantization noise), so that the model can learn its performance under different noise conditions and improve the robustness and generalization ability of the model under different noise conditions.

[0094] Preprocessing of low-density lipoprotein electrophoresis images. Conditional generative adversarial networks (cGANs) can be used to adaptively enhance the contrast and brightness of images during image enhancement, including generators and discriminators. The goal of the generator is to produce visually clearer and higher-contrast images, while the discriminator attempts to distinguish between the original image and the image produced by the generator. The generator network can be used to enhance new low-contrast or low-brightness images. After the image is input to the generator, the output is an image with enhanced contrast and brightness. Optionally, by introducing a self-attention mechanism in cGAN, the model's ability to understand the relationship between different parts of the image is enhanced. The use of denoising autoencoders and conditional generative adversarial networks (cGANs) in LDL electrophoresis image analysis is a new medical image processing method.

[0095] The code for the above process is as follows:

[0096] #Define the generator

[0097] def build_generator(input_shape,label_dim):

[0098] inputs=Input(shape=input_shape)

[0099] labels=Input(shape=(label_dim,))

[0100] #Encoder using self-attention mechanism

[0101] x=Conv2D(64,(3,3),padding='same')(inputs)

[0102] x = BatchNormalization()(x)

[0103] x = LeakyReLU(0.2)(x)

[0104] # Self-attention module

[0105] attention_output = GlobalAveragePooling2D()(x)

[0106] attention_output = Dense(64, activation='relu')(attention_output)

[0107] attention_output = Dense(x.shape[1] * x.shape[2] * x.shape[3], activation='sigmoid')(attention_output)

[0108] attention_output = Reshape((x.shape[1], x.shape[2], 1))(attention_output)

[0109] x = Multiply()([x, attention_output])

[0110] x = Conv2D(128, (3, 3), padding='same')(x)

[0111] x = BatchNormalization()(x)

[0112] x = LeakyReLU(0.2)(x)

[0113] # Merging of conditional vector and feature map

[0114] label_embedding = Dense(128, activation='relu')(labels)

[0115] label_embedding = Reshape((1, 1, 128))(label_embedding)

[0116] x = Add()([x, label_embedding])

[0117] x = Conv2D(1, (3, 3), padding='same', activation='tanh')(x)

[0118] model=Model(inputs=[inputs,labels],outputs=x)

[0119] return model

[0120] #Define the discriminator

[0121] def build_discriminator(input_shape):

[0122] inputs=Input(shape=input_shape)

[0123] x=Conv2D(64,(4,4), strides=(2,2), padding='same')(inputs)

[0124] x = LeakyReLU(0.2)(x)

[0125] x=Conv2D(128,(4,4), strides=(2,2), padding='same')(x)

[0126] x = BatchNormalization()(x)

[0127] x = LeakyReLU(0.2)(x)

[0128] x=Flatten()(x)

[0129] x=Dense(1,activation='sigmoid')(x)

[0130] model=Model(inputs=inputs,outputs=x)

[0131] return model

[0132] #Build cGAN model

[0133] input_shape = (2048, 2048, 1) # Image input size

[0134] label_dim=7#Set 7 different conditional labels, such as: age group, LDL subtype, disease status, gender, lifestyle, eating habits, whether receiving drug treatment

[0135] generator=build_generator(input_shape,label_dim)

[0136] discriminator=build_discriminator((2048,2048,1))

[0137] #Define loss function and optimizer

[0138] discriminator_optimizer=Adam(0.0002,0.5)

[0139] generator_optimizer=Adam(0.0002,0.5)

[0140] discriminator.compile(loss='binary_crossentropy',optimizer=discriminator_optimizer)

[0141] gan=Model(inputs=[generator.input[0],generator.input[1]], outputs=discriminator(generator.output))

[0142] gan.compile(loss='binary_crossentropy',

[0143] optimizer=generator_optimizer)

[0144] #Train GAN

[0145] #noisy_images is a noisy image, labels is the corresponding conditional label

[0146] for epoch in range(epochs):

[0147] noise=np.random.normal(0,1,(batch_size,100))

[0148] generated_images=generator.predict([noisy_images,labels])

[0149] real_labels=np.ones((batch_size,1))

[0150] fake_labels=np.zeros((batch_size,1))

[0151] discriminator.trainable=True

[0152] discriminator.train_on_batch(noisy_images,real_labels)

[0153] discriminator.train_on_batch(generated_images,fake_labels)

[0154] discriminator.trainable=False

[0155] gan_loss=gan.train_on_batch([noisy_images,labels],real_labels)

[0156] print(f"Epoch{epoch+1},GAN loss:{gan_loss}")

[0157] #Use the trained generator for contrast enhancement

[0158] enhanced_images=generator.predict([noisy_images_test,labels])

[0159] In the preprocessing of low-density lipoprotein electrophoresis images, BatchNormalization and Layer Normalization can be applied to automatically normalize the images during image normalization to ensure the stability of subsequent feature extraction and classification, improve the robustness of the model to changes in the input data distribution, and accelerate the model training process. At the same time, deep separable convolution can be used to reduce the amount of calculation and the number of parameters of the model, which is a convolution method widely used in lightweight networks such as MobileNet. Optional, image normalization batch processing mainly includes 4 points: 1. Depthwise separable convolution: In the second convolution block, depthwise separable convolution is used, which decomposes the standard convolution into depthwise convolution (DepthwiseConv2D) and pointwise convolution (PointwiseConv2D), which can reduce the amount of calculation and the number of parameters of the model; 2. Batch Normalization: BatchNormalization is applied after the first convolution block, which normalizes the data according to the mean and variance of each small batch of data, which helps to reduce internal covariate shift; 3. Layer Normalization: Layer Normalization is applied after the second convolution block, which normalizes each channel of each sample. Unlike Batch Normalization, it does not depend on other data in the batch; 4. Mixed precision training: By setting the mixed_float16 policy, mixed precision training is enabled, which can speed up the training process and reduce memory usage. By using batch normalization operations after each convolution layer of the encoder and decoder, internal covariate shift is reduced, the training process is accelerated, and the stability of the model is improved. Among them, learning rate scheduling and early stopping can be used in combination to improve the generalization ability of the model and prevent overfitting.

[0160] The code for image batch normalization is as follows:

[0161] # Enable mixed precision training

[0162] policy=tf.keras.mixed_precision.Policy('mixed_float16')

[0163] tf.keras.mixed_precision.set_policy(policy)

[0164] #Input layer

[0165] input_layer=Input(shape=(2048,2048,3),name='input')

[0166] #The first convolution block uses depth-separable convolution and Batch Normalization

[0167] x=Conv2D(32,(3,3),padding='same',name='conv1')(input_layer)

[0168] x=BatchNormalization(name='bn1')(x)

[0169] x=Activation('relu',name='act1')(x)

[0170] #The second convolution block uses depth-separable convolution and Layer Normalization

[0171] x=DepthwiseConv2D((3,3),padding='same',name='depthwise_conv2')(x)

[0172] x=PointwiseConv2D(64,(1,1),name='pointwise_conv2')(x)

[0173] x=LayerNormalization(name='ln2')(x)

[0174] x=Activation('relu',name='act2')(x)

[0175] #Pooling layer

[0176] x=MaxPooling2D(pool_size=(2,2),name='maxpool')(x)

[0177] # Flatten the layers

[0178] x=Flatten(name='flatten')(x)

[0179] #Fully connected layer

[0180] x=Dense(128,name='fc1')(x)

[0181] x=BatchNormalization(name='bn_fc1')(x)

[0182] x=Activation('relu',name='act_fc1')(x)

[0183] #Output layer

[0184] output_layer=Dense(1,activation='sigmoid',name='output')(x)

[0185] #Build the model

[0186] model=Model(inputs=input_layer,outputs=output_layer)

[0187] # Compile the model

[0188] model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy'])

[0189] #Print model structure

[0190] #model.summary()

[0191] The pre-trained Vision Transformer (ViT) model can be used to extract key features from the pre-processed LDL electrophoresis images. ViT can capture global and local information of the image and is suitable for processing complex image data. Key features include: color features, texture features, shape features, size features, position features, context features (referring to the relative position of each strip); these features will be used as input for subsequent classification models to predict diseases. Combined with deep learning, feature extraction can extract high-level features from images in an automated way, avoiding the limitations of traditional manually designed features. Optionally, high-level features refer to: strip morphology, size, complex texture, spatial relationship, etc.

[0192] Vision Transformer (ViT) can capture both global and local information of an image. Global information may include: the overall layout of the image, such as the overall distribution pattern of LDL bands in an electrophoresis image, including whether the bands tend to be concentrated in a certain area of ​​the image or are evenly distributed throughout the image; the overall density of LDL, such as the number of LDL bands in the entire image, which can be used to extract information about the overall density of LDL and thus determine the risk of disease (such as cardiovascular disease); image contrast and brightness: capturing the contrast and brightness levels of the entire image, which is very important for distinguishing LDL bands from the background; and the overall migration distance of LDL: during electrophoresis, the size and charge characteristics of LDL bands can be reflected by identifying the overall distance that LDL bands migrate on the gel. The local information mainly includes: Morphological characteristics of LDL bands: capturing the local morphological characteristics of LDL bands, such as the shape of the bands (circular, oval or other irregular shapes) and edge clarity; Size changes of LDL bands: identifying the size changes of LDL bands, which may be related to the subtype classification of LDL; Local density of LDL bands: extracting the density information of LDL bands in a specific area, which helps to analyze the local aggregation of LDL during electrophoresis; Distribution pattern of LDL bands: ViT can identify the local distribution pattern of LDL bands in the image, such as whether the bands form specific patterns or aggregates in certain areas; Overlap and interaction of LDL bands: In electrophoresis images, LDL bands may overlap or interact. ViT can capture these local interactions, which is very important for understanding the physical properties of LDL. Local contrast and brightness changes: ViT can identify the contrast and brightness changes in specific areas of the image, which helps to distinguish LDL bands from the background and identify possible local staining or damage. At the same time, we can improve vit and enhance its performance in processing LDL electrophoresis images, so as to distinguish the healthy sample features and patient sample features of electrophoresis images: for example:

[0193] 1. Define model components

[0194] Custom Positional Encoding: Defines a positional encoding module to add position information to each patch of the image, which helps the model understand the spatial structure in the image.

[0195] MultiScale Attention: A multi-scale self-attention module is defined, which allows the model to capture image features at different scales, enhancing the perception of local details and global context.

[0196] Custom ViT model (CustomViTModel): This is the main part of the model, combining patch embedding, positional encoding, Transformer Encoder, multi-scale self-attention and the final classifier.

[0197] Useful features are extracted from LDL electrophoresis images and used for classification tasks. Through training and evaluation, the model is expected to achieve a certain accuracy.

[0198] Through custom position encoding, each image patch is assigned a unique position identifier, which helps the model understand the spatial structure in the image and the positional relationship of each patch. Capture the spatial information of the relative position and distribution of strips in the image. Through the multi-scale self-attention mechanism, image features are captured at different scales. Local details (small textures and edges), global patterns (overall shape and structure size). By splitting the image into multiple patches, each patch is then converted to a high-dimensional vector through a linear layer (i.e., patch embedding). Information from different color channels can be fused together. Optionally, add a [CLS] token to aggregate the semantic information of the entire image. Such as the overall description of the image, category, patient gender, etc.

[0199] The relevant codes for the above feature extraction implementation are as follows:

[0200] import torch

[0201] import torch.nn as nn

[0202] import torch.nn.functional as F

[0203] from transformers import ViTConfig

[0204] import math

[0205] from torch.utils.data import Dataset,DataLoader

[0206] from torchvision import transforms

[0207] from PIL import Image

[0208] #Setting up the device

[0209] device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

[0210] # Custom positional encoding

[0211] class CustomPositionalEncoding(nn.Module):

[0212] def __init__(self, dim, max_len = 2048):

[0213] super(CustomPositionalEncoding, self).__init__()

[0214] self.encoding = torch.zeros(max_len, dim)

[0215] position = torch.arange(0, max_len,

[0216] dtype = torch.float).unsqueeze(1)

[0217] div_term = torch.exp(torch.arange(0, dim, 2).float() * (-torch.log(torch.tensor(10000.0)) / dim))

[0218] self.encoding[:, 0::2] = torch.sin(position * div_term)

[0219] self.encoding[:, 1::2] = torch.cos(position * div_term)

[0220] self.encoding = self.encoding.unsqueeze(0)

[0221] def forward(self, x):

[0222] return self.encoding[:, :x.size(1)].detach()

[0223] # Multi-scale self-attention mechanism

[0224] class MultiScaleAttention(nn.Module):

[0225] def__init__(self,dim,num_heads=8,scale_factor=2):

[0226] super(MultiScaleAttention,self).__init__()

[0227] self.scale_factor=scale_factor

[0228] self.num_heads=num_heads

[0229] self.dim=dim

[0230] self.head_dim=dim / / num_heads

[0231] self.query=nn.Linear(dim,dim)

[0232] self.key=nn.Linear(dim,dim)

[0233] self.value=nn.Linear(dim,dim)

[0234] self.out=nn.Linear(dim,dim)

[0235] def forward(self,x):

[0236] B,N,C=x.shape

[0237] q=self.query(x).reshape(B,N,self.num_heads,self.head_dim).permute(0,2,1,3)

[0238] k=self.key(x).reshape(B,N,self.num_heads,self.head_dim).permute(0,2,1,3)

[0239] v=self.value(x).reshape(B,N,self.num_heads,self.head_dim).permute(0,2,1,3)

[0240] k_down = k[:, :, ::self.scale_factor, :].reshape(B, -1, self.num_heads, self.head_dim).permute(0, 2, 1, 3)

[0241] v_down = v[:, :, ::self.scale_factor, :].reshape(B, -1, self.num_heads, self.head_dim).permute(0, 2, 1, 3)

[0242] dots = torch.matmul(q, k.transpose(-2, -1)) / math.sqrt(self.head_dim)

[0243] dots_down = torch.matmul(q, k_down.transpose(-2, -1)) / math.sqrt(self.head_dim)

[0244] attention = dots + dots_down

[0245] attention = F.softmax(attention, dim=-1)

[0246] out = torch.matmul(attention, v) + torch.matmul(attention[:, :, :N / / self.scale_factor, :], v_down)

[0247] out = out.transpose(1, 2).reshape(B, N, C)

[0248] return self.out(out)

[0249] # Custom ViT model

[0250] class CustomViTModel(nn.Module):

[0251] def __init__(self, config):

[0252] super(CustomViTModel, self).__init__()

[0253] self.patch_embedding = nn.Conv2d(config.image_size[0] * config.image_size[0], config.hidden_size, kernel_size = 1)

[0254] self.position_embedding = CustomPositionalEncoding(config.hidden_size,

[0255] config.max_position_embeddings)

[0256] self.encoder = nn.TransformerEncoderLayer(d_model = config.hidden_size, nhead = config.num_attention_heads,

[0257] dim_feedforward = config.intermediate_size,

[0258] dropout = config.hidden_dropout_prob)

[0259] self.multi_scale_attention = MultiScaleAttention(config.hidden_size, config.num_attention_heads)

[0260] self.final_layer_norm = nn.LayerNorm(config.hidden_size, eps = config.layer_norm_eps)

[0261] self.classifier = nn.Linear(config.hidden_size, 2) # Assume there are two classes: normal people and cardiovascular patients

[0262] def forward(self, pixel_values):

[0263] #Convert pixel_values to embeddings

[0264] batch_size, num_channels, height, width = pixel_values.shape

[0265] embeddings = self.patch_embedding(pixel_values) # (batch_size, hidden_size, height*width)

[0266] embeddings = embeddings.reshape(batch_size, -1, embeddings.shape[1]) # (batch_size, height*width, hidden_size)

[0267] embeddings += self.position_embedding(embeddings)

[0268] embeddings = self.encoder(embeddings.permute(1, 0, 2)).permute(1, 0, 2)

[0269] embeddings = self.multi_scale_attention(embeddings)

[0270] embeddings = self.final_layer_norm(embeddings)

[0271] return self.classifier(embeddings.mean(dim = 1)) # 取平均池化作为图像的全局表示

[0272] # Custom ViT configuration

[0273] class CustomViTConfig:

[0274] def __init__(self, **kwargs):

[0275] self.hidden_size = kwargs.get("hidden_size", 768)

[0276] self.max_position_embeddings = kwargs.get("max_position_embeddings", 1024)

[0277] self.num_attention_heads = kwargs.get("num_attention_heads", 12)

[0278] self.intermediate_size = kwargs.get("intermediate_size", 3072)

[0279] self.hidden_dropout_prob = kwargs.get("hidden_dropout_prob", 0.1)

[0280] self.image_size = kwargs.get("image_size", (3, 224, 224))

[0281] self.layer_norm_eps = kwargs.get("layer_norm_eps", 1e - 12)

[0282] # Instantiate the configuration and the model

[0283] config = CustomViTConfig()

[0284] model = CustomViTModel(config).to(device)

[0285] # Define the dataset

[0286] class LDLDataset(Dataset):

[0287] def __init__(self, image_paths, labels, transform = None):

[0288] self.image_paths = image_paths

[0289] self.labels = labels

[0290] self.transform = transform

[0291] def __len__(self):

[0292] return len(self.image_paths)

[0293] def__getitem__(self,idx):

[0294] image_path=self.image_paths[idx]

[0295] label = self.labels[idx]

[0296] image=Image.open(image_path).convert('RGB')

[0297] if self.transform:

[0298] image = self.transform(image)

[0299] return{"pixel_values":image,"labels":torch.tensor(label,dtype=torch.long)}

[0300] #Data preprocessing

[0301] transform=transforms.Compose([

[0302] transforms.Resize((224,224)),

[0303] transforms.ToTensor(),

[0304] transforms.Normalize(mean=[0.485,0.456,0.406],std=[0.229,0.224,0.225]) ])

[0306] # Load data

[0307] image_paths = 'path / to / samples' #image path list

[0308] labels = [0]*500+[1]*500 #label list, 0 represents normal people, 1 represents cardiovascular and cerebrovascular patients

[0309] dataset=LDLDataset(image_paths,labels,transform=transform)

[0310] dataloader = DataLoader(dataset, batch_size=32, shuffle=True)

[0311] # Train and evaluate the model

[0312] optimizer = torch.optim.Adam(model.parameters(), lr=1e-5)

[0313] criterion = nn.CrossEntropyLoss()

[0314] num_epochs = 5

[0315] for epoch in range(num_epochs):

[0316] model.train()

[0317] total_loss = 0

[0318] for batch in dataloader:

[0319] pixel_values = batch['pixel_values'].to(device)

[0320] labels = batch['labels'].to(device)

[0321] optimizer.zero_grad()

[0322] outputs = model(pixel_values=pixel_values)

[0323] loss = criterion(outputs, labels)

[0324] loss.backward()

[0325] optimizer.step()

[0326] total_loss += loss.item()

[0327] print(f'Epoch {epoch}, Loss: {total_loss / len(dataloader)}') # Evaluate the model

[0328] model.eval()

[0329] correct=0

[0330] total=0

[0331] with torch.no_grad():

[0332] for batch in dataloader:

[0333] pixel_values=batch['pixel_values'].to(device)

[0334] labels=batch['labels'].to(device)

[0335] outputs=model(pixel_values=pixel_values)

[0336] _,predicted=torch.max(outputs,dim=1)

[0337] total+=labels.size(0)

[0338] correct+=(predicted==labels).sum().item()

[0339] accuracy = correct / total

[0340] print(f'Accuracy:{accuracy:.2f}')

[0341] The disease prediction training model can be constructed by using the recursive feature elimination (RFE) method for feature selection, thereby recursively removing the key features of healthy negative samples, and then using the selected features to train the XGBoost classification model, which can handle high-dimensional data and has a strong feature selection capability, reflecting the innovation of feature selection and model training. A variety of evaluation indicators (such as accuracy, F1 score, AUC-ROC) can be used to comprehensively evaluate the performance of the model.

[0342] The specific steps of feature selection using the recursive feature elimination (RFE) method can be:

[0343] Before inputting the features into RFE, the data is preprocessed by standardization and normalization to ensure that all features are at the same scale.

[0344] Select a base model for RFE. In this embodiment, the XGBoost classification model is used as the final classifier, so XGBoost can be selected as the base model because it has good regularization ability and feature importance evaluation function.

[0345] Use XGBoost to train the initial feature set and identify the least important features; recursively remove a certain proportion of the least important features and retrain the model on the remaining features; repeat the above process until the predetermined number of features is reached or the performance is no longer significantly improved. After the feature set recursion (RFE) is completed, select the remaining features as the final feature set. When recursively removing unimportant features, you can refer to the unimportant feature set or the feature importance score according to XGBoost. Use the selected features to train the XGBoost classification model to identify unimportant features, and make predictions and evaluate the accuracy of the model on the test set. The combination of the recursive feature elimination (RFE) method and the XGBoost classification model can handle high-dimensional data and has a strong feature selection capability, reflecting the innovation of feature selection and model training.

[0346] After extracting the features, the disease prediction model is constructed. Optionally, deep learning + centralized learning joint modeling is used to combine the powerful feature extraction capabilities of deep learning and the interpretability of traditional machine learning models, and the prediction performance and robustness of the model are improved through the stacking method.

[0347] The initial construction process of the disease prediction model includes two main steps: feature selection and building a classification model. Optionally, a combination of tree-based feature selection and XGBoost classification model, which can handle high-dimensional data and has strong feature selection capabilities, also performs well in classification tasks. Ensemble learning can combine the prediction results of multiple models to improve the overall prediction accuracy. Optionally, use random forest and gradient boosting machine as the other two ensemble learners, and then use the stacking method to fuse the prediction results of random forest and gradient boosting machine.

[0348] The specific process is as follows: StandardScaler is used to standardize the extracted features to ensure that all features have zero mean and unit variance; the dataset is divided into training set and test set. Support vector machine (SVM) is used as the base model, and a specific number of the most important features are selected by recursive feature elimination (RFE) method. A deep learning model is built using Keras library, which contains multiple fully connected layers, batch normalization layers, and Dropout layers to prevent overfitting. The model is compiled using Adam optimizer and binary cross entropy loss function, and trained on the training set. The trained deep learning model is used to predict on the test set to generate prediction results. Random forest and gradient boosting machine are trained as base learners, and the prediction results of the deep learning model are used as additional features. The prediction results of random forest, gradient boosting machine and deep learning model are combined using StackingClassifier to form an integrated learning model. The stacking model is trained on the training set and fine-tuned on the validation set. The stacking model is used to predict on the test set, and the performance of the model is evaluated using accuracy, classification report, ROC curve and AUC-ROC value.

[0349] The code for the above process is as follows:

[0350] from sklearn.feature_selection import RFE

[0351] from sklearn.svm import SVC

[0352] from sklearn.preprocessing import StandardScaler

[0353] from sklearn.ensemble import RandomForestClassifier,GradientBoostingClassifier,StackingClassifier

[0354] from sklearn.metrics import accuracy_score,classification_report,roc_auc_score,roc_curve

[0355] from sklearn.linear_model import LogisticRegression

[0356] import matplotlib.pyplot as plt

[0357] import numpy as np

[0358] from tensorflow.keras.models import Sequential

[0359] from tensorflow.keras.layers import Dense,Dropout,BatchNormalization

[0360] from tensorflow.keras.optimizers import Adam

[0361] #Data Standardization

[0362] scaler = StandardScaler()

[0363] X_scaled=scaler.fit_transform(extracted_features)

[0364] #Use SVM as the base model for RFE

[0365] svm = SVC (kernel = "ploy", C = 25) # Use polynomial kernel SVM, innovative regularization parameter C = 25

[0366] rfe = RFE (estimator = svm, n_features_to_select = 6) # Select the top 6 most important features

[0367] rfe.fit(X_train,y_train)

[0368] #Use the selected features

[0369] X_train_selected=X_train[:,rfe.support_]

[0370] X_test_selected=X_test[:,rfe.support_]

[0371] #Build a deep learning model,

[0372] Set the first layer of Dense Layers to 156 units, the second layer to 96 units, and the third layer to 38 units using the relu activation function;

[0373] The application ratio after each Dense layer is set to 0.6 to prevent overfitting;

[0374] When using the Adam optimizer, the learning rate is set to 0.061, which makes the model converge quickly and have good performance;

[0375] When training the model, we set the number of simulations epochs = 83 and the number of data samples used to train the model before each gradient update batch_size = 37.

[0376] model=Sequential([

[0377] Dense(156,activation='relu',input_shape=(X_train_selected.shape[1],)),

[0378] BatchNormalization(),

[0379] Dropout(0.6),

[0380] Dense(96,activation='relu'),

[0381] BatchNormalization(),

[0382] Dropout(0.6),

[0383] Dense(38,activation='relu'),

[0384] Dense(1,activation='sigmoid') ])

[0386] # Compile the model

[0387] model.compile(optimizer=Adam(learning_rate=0.061),loss='binary_crossentropy',metrics=['accuracy'])

[0388] #Train model

[0389] model.fit(X_train_selected,y_train,epochs=83,batch_size=37,validation_split=0.2)

[0390] #Use deep learning model to make predictions

[0391] deep_learning_predictions=model.predict(X_test_selected).flatten()

[0392] #Define the base learner and set the number of decision trees (n_estimators:) to 120.

[0393] The seed of the random number generator (random_state) is set to 61.

[0394] rf_clf=RandomForestClassifier(n_estimators=120,random_state=61)

[0395] gb_clf=GradientBoostingClassifier(n_estimators=120,random_state=61)

[0396] #Training base learner

[0397] rf_clf.fit(X_train_selected,y_train)

[0398] gb_clf.fit(X_train_selected,y_train)

[0399] #Create a stacked model

[0400] stack_clf=StackingClassifier(

[0401] estimators = [

[0402] ('rf',rf_clf),

[0403] ('gb',gb_clf),

[0404] ('deep_learning',model)#Use the deep learning model as one of the base learners

[0405] ],

[0406] final_estimator=LogisticRegression(),

[0407] cv=5 )

[0409] #Training stacked model

[0410] stack_clf.fit(X_train_selected,y_train)

[0411] # Make predictions on the test set

[0412] y_pred_stack=stack_clf.predict(X_test_selected)

[0413] #Evaluate the performance of the stacked model

[0414] accuracy=accuracy_score(y_test,y_pred_stack)

[0415] print(f"Accuracy of Stacking Classifier:{accuracy}")

[0416] #Print classification report

[0417] print("Classification Report:\n",classification_report(y_test,y_pred_stack))

[0418] # Calculate AUC-ROC

[0419] y_pred_proba_stack=stack_clf.predict_proba(X_test_selected)[:,1]

[0420] auc_roc=roc_auc_score(y_test,y_pred_proba_stack)

[0421] #Draw the ROC curve

[0422] fpr,tpr,_=roc_curve(y_test,y_pred_proba_stack)

[0423] plt.figure()

[0424] plt.plot(fpr,tpr,label=f"Stacking ROC curve(area={auc_roc:.2f})",color='blue')

[0425] plt.plot([0,1],[0,1],'k--')

[0426] plt.xlabel('False Positive Rate')

[0427] plt.ylabel('True Positive Rate')

[0428] plt.title('ROC Curve for Stacking Classifier')

[0429] plt.legend(loc="lower right")

[0430] plt.show()

[0431] The prediction results of random forest, gradient boosting machine and deep learning model are combined by stacking method to improve the overall prediction accuracy. In the RFE process, the prediction results of these integrated learning models are used as features and further provided to the XGBoost classification model, thereby enhancing the accuracy of feature selection and the prediction performance of the model. In the RFE process, not only the feature importance of the traditional machine learning model is considered, but also the advanced features extracted by the deep learning model are combined, thereby improving the quality of feature selection. Optionally, in the RFE process, the selected features need to be verified on the data set processed by data enhancement to ensure the robustness of the feature selection process. By introducing the self-attention mechanism and deep separable convolution in the deep learning model, the quality of feature extraction is improved, so that RFE can be selected from a richer and higher quality feature set. The embodiment of the present invention combines deep learning and traditional machine learning techniques for the analysis and disease prediction of LDL electrophoresis images, and adopts mixed precision training to accelerate the training process of the model and reduce memory usage, which can improve training efficiency and model performance.

[0432] In summary, the present application provides a model and system for low-density lipoprotein electrophoresis image recognition and disease prediction:

[0433] Through automated image recognition technology and deep learning models, the efficiency and accuracy of LDL typing detection have been improved. And the cost of LDL typing detection has been reduced, making this technology affordable and widely used by more medical institutions. By combining deep learning and traditional machine learning techniques, the reliability of disease prediction has been improved, providing a powerful tool for clinical diagnosis. This solution can help doctors quickly determine the risk of patients and assist in the formulation of personalized treatment plans. It also shows how to apply the latest artificial intelligence technology to medical image analysis, promoting the innovation and development of medical technology. By providing more accurate disease predictions, it helps doctors make more accurate clinical decisions. Through automated and precise testing processes, the utilization efficiency of medical resources has been improved, especially in resource-limited environments.

[0434] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. Based on low-density lipoprotein electrophoresis image recognition and disease prediction model, it is characterized by: include: Collect diseased positive samples and healthy negative samples of low-density lipoprotein electrophoresis images; Extract key features of preprocessed low-density lipoprotein electrophoresis images; Collecting multiple sets of disease prediction training data; each set of disease prediction training data includes a positive sample of a patient, at least one key feature, one or more disease types, and an identifier for indicating whether the patient suffers from each disease type; and obtaining a disease prediction training model by training the multiple sets of disease prediction training data; Collecting multiple sets of disease prediction verification data; each set of disease prediction verification data includes a healthy negative sample of the patient, at least one key feature, and one or more disease types; the multiple sets of disease prediction verification data are tested by the disease prediction training model respectively; When the output is identified as suffering from each disease type, the key features of the healthy negative samples are recursively removed until the disease prediction training model outputs an identification of not suffering from each disease type, and the removed key features are collected as unimportant features; finally, an unimportant feature set of each disease type is obtained; Collect multiple sets of disease prediction optimization data; each set of the disease prediction optimization data includes healthy negative samples of patients, key features after recursively removing at least one unimportant feature each time, one or more disease types, and identifiers used to indicate that the patient does not suffer from each disease type; train the disease prediction training model using the multiple sets of disease prediction optimization data.

2. The low-density lipoprotein electrophoresis image recognition and disease prediction model according to claim 1, characterized in that: The basis for recursively removing each key feature is to preferentially delete the same features between the healthy negative sample and the diseased positive sample of the same disease type, and the same features between the healthy negative samples of the same disease type.

3. The low-density lipoprotein electrophoresis image recognition and disease prediction model according to claim 1, characterized in that: The key features of the pre-processed low-density lipoprotein electrophoresis image are extracted using a pre-trained feature extraction model; the key features include any one or more of color, texture, shape, size, position, and context.

4. The low-density lipoprotein electrophoresis image recognition and disease prediction model according to claim 1, characterized in that: The preprocessing of the low-density lipoprotein electrophoresis image includes any one or more operations of denoising, image enhancement and image normalization.

5. The low-density lipoprotein electrophoresis image recognition and disease prediction model according to claim 1, characterized in that: After extracting the key features of the pre-processed low-density lipoprotein electrophoresis image, the key features are pre-processed; pre-processing the key features includes standardization and normalization operations.

6. The low-density lipoprotein electrophoresis image recognition and disease prediction model according to claim 1, characterized in that: The key features are recursively removed, and features are selected using a recursive feature elimination method.

7. The low-density lipoprotein electrophoresis image recognition and disease prediction model according to claim 1, characterized in that: When the plurality of groups of the disease prediction training data are trained to obtain a disease prediction training model, the disease prediction training model is trained through the following steps: Build deep learning models; compile and train deep learning models; train random forests and gradient boosting machines as base learners and use the predictions of deep learning models as additional features; The prediction results of the random forest, gradient boosting machine and deep learning model are combined using StackingClassifier to form an integrated learning model as the disease prediction training model.

8. Based on low-density lipoprotein electrophoresis image recognition and disease prediction system, characterized in that: include: Image sample collection module: collects diseased positive samples and healthy negative samples of low-density lipoprotein electrophoresis images; Image feature extraction module: extract key features of the pre-processed low-density lipoprotein electrophoresis image; Disease prediction training module: collect multiple sets of disease prediction training data; each set of disease prediction training data includes a positive sample of a disease, at least one key feature, one or more disease types, and an identifier for indicating whether the patient suffers from each disease type; a disease prediction training model is obtained by training multiple sets of the disease prediction training data; Disease prediction test module: collect multiple sets of disease prediction verification data; each set of disease prediction verification data includes a healthy negative sample of the patient, at least one key feature, and one or more disease types; the multiple sets of disease prediction verification data are tested by the disease prediction training model respectively; Disease feature identification module: when the output is identified as suffering from each disease type, recursively remove the key features of the healthy negative sample until the disease prediction training model outputs an identification of not suffering from each disease type, and collects the removed key features as unimportant features; finally, an unimportant feature set of each disease type is obtained; Disease prediction optimization module: collect multiple groups of disease prediction optimization data; each group of the disease prediction optimization data includes healthy negative samples of patients, key features after recursively removing at least one unimportant feature each time, one or more disease types, and identifiers used to indicate that the patient does not suffer from each disease type; the disease prediction training model is trained using multiple groups of the disease prediction optimization data.

9. An electronic device comprising a memory, a processor, and a computer program running on the processor, characterized in that: When the processor executes the computer program, the low-density lipoprotein electrophoresis image recognition and disease prediction model according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the low-density lipoprotein electrophoresis image recognition and disease prediction model according to any one of claims 1 to 7 is implemented.

Images