Small sample cell image classification method based on texture feature extraction and fusion
The texture features of cell images were extracted through LBP and GLCM algorithms, and the linear weighted fusion and KNN algorithms were solved, and the accuracy of cell image classification was improved.
Patent Information
- Application Number
- CN202510413586.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-03
- Publication Date
- 2025-07-11
AI Technical Summary
Existing cell image classification methods are difficult to effectively integrate multiple features in a small sample environment, resulting in a degradation of classification performance, especially relying on a single feature can easily lead to overfitting.
The texture features were extracted separately by LBP and GLCM algorithms, and the depth feature vector was generated by pre-training the neural network, and the optimal weight was determined through grid search for linear weighting fusion. Finally, the KNN algorithm was used for cell image classification.
The model's characterization ability and classification accuracy of cell images are improved, and the classification performance under small sample conditions is significantly improved.
Smart Images
Figure CN120299038A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of small-sample cells, and particularly relates to a small-sample cell image classification method based on texture feature extraction and fusion. Background Art
[0002] Cell image classification is an important task in biomedical research and is widely used in fields such as tumor detection, disease diagnosis, and drug development. Traditional cell classification methods mainly rely on manual observation and qualitative analysis under a microscope. This method is not only time-consuming but also limited by the experience and professionalism of operators, making it difficult to ensure the stability and accuracy of classification results. Therefore, how to improve the efficiency and accuracy of cell classification through automated technology has become the focus of research. Based on this, the application of computer vision and deep learning technology in cell image classification has gradually become a research hotspot.
[0003] However, traditional neural network models usually require a large amount of data for effective training. Especially in the case of rare or early diseases, it is often difficult to obtain a sufficient number of samples. For this reason, researchers have developed various coping strategies, such as data augmentation, transfer learning, and feature extraction, to improve the model performance under limited data conditions. Among them, the feature extraction method significantly reduces the dependence on a large amount of data by extracting key texture, shape, and color features from limited samples, thereby effectively improving the model performance. In addition, compared with other deep learning models, the feature extraction method only needs to store the extracted features, significantly reducing the memory consumption and enabling the model to run more efficiently under limited computing resources.
[0004] Although feature extraction can effectively extract key features in cell images, such as texture, shape, and color, there are still problems in the current research that various features cannot be effectively fused. The limitation of single feature leads to the difficulty for the model to comprehensively understand the complexity of cells, reducing the classification performance. Especially in a small-sample environment, over-reliance on a single feature is likely to cause overfitting. Therefore, it is crucial to improve the fusion effect between features. Summary of the Invention
[0005] Object of the Invention: The object of the present invention is to provide a small-sample cell image classification method based on texture feature extraction and fusion. By adopting an effective feature fusion mechanism, it makes full use of the complementary information between different features, improves the model's representation ability and classification accuracy for cell images, and solves the problems existing in the background art.
[0006] Technical Solution: A small-sample cell image classification method based on texture feature extraction and fusion according to the present invention includes the following steps:
[0007] (1) Preprocess the cell image dataset; perform normalization operations, including size adjustment, random cropping, channel conversion, and normalization, to generate input images of a unified specification;
[0008] (2) Use the LBP algorithm and the GLCM algorithm to extract the texture features of the preprocessed images respectively, generating LBP texture images and GLCM texture images;
[0009] (3) Adopt a pre-trained neural network model to perform feature encoding on the LBP texture images and GLCM texture images obtained in step (2) respectively, and output deep feature vectors with a dimension of 1536;
[0010] (4) Determine the optimal weights through grid search, linearly weight and fuse the deep feature vectors of LBP and GLCM to generate fused feature vectors;
[0011] (5) Based on the fused feature vectors, calculate the Euclidean distance between the image to be classified and the sample images through the KNN algorithm to complete the determination of the cell image category;
[0012] (6) Display the classification results through a graphical interface, supporting the adjustment of the number of samples and the visualization of the classification accuracy.
[0013] Further, in step (1), the preprocessing includes: size adjustment, random cropping, channel conversion, and normalization operations to generate input images of a unified specification.
[0014] Further, in step (2), the implementation method of the LBP algorithm is: calculate the gray-scale contrast between the central pixel and the surrounding pixels in a 3×3 pixel neighborhood, generate an eight-bit binary code and convert it into a decimal value. The formula is:
[0015]
[0016] where g c represents the pixel value of the central pixel point, and g p represents the pixel value of the neighborhood pixel point.
[0017] Further, in step (2), the GLCM algorithm extracts the Entropy feature, and the calculation formula is:
[0018] P(i,j|d,θ)={(x,y)|f(x,y)=i,f(x+dx,y+dx)=j;x,y=0,1,…N-1}
[0019] where d represents the relative position of the pixel quantity; θ generally considers four directions 0°, 45°, 90°, 135°; i,j=0,1,2,…,,L-1; (x,y) is the pixel coordinate in the image, and L is the number of gray levels of the image.
[0020] Further, in step (3), the pre-trained network of the Dino model is adopted to output a 1536-dimensional feature vector before the SoftMax layer.
[0021] Further, in step (4), the grid search is as follows: a combination of parameter candidates is formulated through a predetermined step size or search range, where the candidate combination covers all combinations of the parameters to be searched; the generated combination is applied to the model, and the performance of the model is optimized by adjusting the weights of the feature vectors.
[0022] Further, in step (5), the determination method of KNN classification is: calculate the sum of the Euclidean distances between the image to be classified and each class of samples, and select the class with the smallest sum as the prediction result.
[0023] Further, in step (6), the graphical interface is implemented based on the PyQt5 framework.
[0024] An electronic device according to the present invention includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is loaded into the processor, it implements any one of the methods for classifying small-sample cell images based on texture feature extraction and fusion.
[0025] A storage medium according to the present invention stores a computer program, and when the computer program is executed by a processor, it implements any one of the methods for classifying small-sample cell images based on texture feature extraction and fusion.
[0026] Beneficial effects: Compared with the prior art, the present invention has the following remarkable advantages: in the texture feature extraction of the present invention, texture feature images are extracted from the preprocessed images, and LBP and GLCM feature images are extracted from the images; then the texture images extracted by the texture feature extraction are input into the deep feature extraction module to obtain the feature encoding before the SoftMax layer; then, the feature encodings extracted from LBP and GLCM respectively in the previous step are input into the feature fusion module to fuse the two feature encodings; finally, the distance between the feature encoding extracted and fused from the unclassified image and the sample image is calculated in the classification module, and the KNN classification is performed on the sample to be classified according to the Euclidean distance; by adopting an effective feature fusion mechanism, the present invention makes full use of the complementary information between different features, improves the representation ability of the model for cell images and the classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 is a flowchart of the present invention;
[0028] Figure 2 is the extraction of the number of pixel pairs of the gray-level co-occurrence matrix of the present invention;
[0029] Figure 3 is the feature output of the input image before the SoftMax layer extracted by the neural network of the present invention;
[0030] Figure 4 is the feature extraction and linear fusion process of the present invention;
[0031] Figure 5 is the grid search for the best weight of the present invention;
[0032] Figure 6 is the grid search and feature fusion of the present invention;
[0033] Figure 7 is the similarity calculation between the image to be classified and the sample feature vector of the present invention. Detailed implementation manners
[0034] The technical solution of the present invention will be further described below with reference to the accompanying drawings.
[0035] As Figure 1 shown, an embodiment of the present invention provides a small-sample cell image classification method based on texture feature extraction and fusion, including the following steps:.
[0036] Step 1, stretch the images to a unified pixel size, and randomly select several datasets and test sets from the dataset, including the following steps:
[0037] (1-1) Dataset division: Divide different types of cells. The minimum number of cell types in the same dataset is 3, and the maximum is 28, generally 4 to 6. The present invention performs cell feature fusion and classification under the condition of small samples. Therefore, the number of samples selected in one sample is small, in the range of 1 to 8. The samples are randomly selected from each category in the dataset. Since the samples drawn will affect the specific classification effect due to the degree of representativeness of the category to which the sample belongs, the steps of sample extraction and classification will be repeated 100 times, and the mean value is taken as the final classification accuracy.
[0038] (1-2) Picture size standardization: This step of image size standardization is to stretch the size of the image and convert it into an appropriate number of channels for subsequent input into the model, including size stretching operation, image cropping operation, image RGB conversion operation, vectorization operation, and standardization operation.
[0039] (a) Size stretching operation: Stretch the size of the original image to 256*256 to enlarge the size of the image.
[0040] (b) Image cropping operation: Randomly crop the image to crop out a sub-image of 224*224, enhance the dataset, increase the number of pictures, and improve the accuracy of training.
[0041] (c) Image RGB operation: Modify the number of channels of the image to 3. For example, convert a single-channel grayscale image to a 3-channel grayscale image, where the R, G, and B channels are all copies of the original grayscale image channel. For a 4-channel (RGBA) PNG image, the alpha channel is ignored. The purpose of this operation is to make the image meet the requirements of the model input image.
[0042] (d) Vectorization operation: Transfer the image from memory to the GPU video memory.
[0043] (e) Normalization operation: Modify each channel of the original image from integer type to floating-point type, and at the same time limit the size of all data between 0 and 1. The purpose of this operation is to make the images have a similar distribution, correct the differences between images, facilitate the convergence of subsequent training, and reduce the inference error.
[0044] Step 2: Use the LBP and GLCM algorithms to extract texture features from the image respectively. The texture features selected in the present invention are the Entropy features in LBP and GLCM. The LBP feature extraction uses the LBP operator provided by OpenCV, and the Entropy feature extraction of GLCM uses the open-source code fast_glcm for extraction.
[0045] LBP: Adopt the original LBP method, and the operation method of the texture feature image is as follows:
[0046] (2-1) Divide the original image into blocks with a side length of 3 according to pixel squares, and perform the following operations on each block;
[0047] (2-2) In each smallest unit block, compare the base point element with the surrounding elements to obtain the value of the calculated texture image;
[0048] (2-3) Multiply the value of each surrounding element by a specific weight value to obtain an eight-bit number, and convert it to a decimal number, which is the grayscale value of the center point.
[0049] The calculation formula of LBP is as follows:
[0050]
[0051] where g c represents the pixel value of the central pixel point, and g p represents the pixel value of the neighboring pixel point.
[0052] GLCM: The principle of GLCM is as follows: For the gray-level co-occurrence matrix T(N×N), first define the direction and the step size in pixels. The direction is generally 0°, 45°, 90°, and 135°, and M(i,j) is defined as the pixels with gray levels i and j appearing at the same point, and count the points with the same direction and step size frequency. N is the number of gray-level divisions, usually set to 8. In the horizontal direction, the gray-level co-occurrence matrix of the pixels in the horizontal and vertical directions is called the symmetric co-occurrence matrix. Conversely, the pixels in a single direction are called the asymmetric co-occurrence matrix. The calculation formula of GLCM is as follows:
[0053] P(i,j|d,θ)={(x,y)|f(x,y)=i,f(x+dx,y+dx)=j;x,y=0,1,…N-1} ⑵
[0054] Among them, d represents the relative position of the number of pixels; θ generally considers four directions 0°, 45°, 90°, 135°; i,j=0,1,2,…,,L-1;
[0055] (x,y) are the pixel coordinates in the image, and L is the number of gray levels of the image.
[0056] The feature extraction of the GLCM gray-level co-occurrence matrix is as follows:
[0057] (2-4) Convert the original image into a single-channel image by channel narrowing, select specific directions (0°, 45°, 90°, and 135°) to traverse each pixel pair in the image, and calculate the values of the elements at the corresponding positions in the matrix. See Figure 2 .
[0058] (2-5) Normalize the GLCM matrix, calculate the corresponding features for each element in the matrix, and obtain the feature values of the pixels. After calculating the above GLCM matrix, calculate the corresponding values according to the formula in Table 1 below.
[0059] Table 1 Features and Their Calculation Formulas
[0060]
[0061] The above features can represent the characteristics of the image in different aspects. For example, energy can be used to reflect the distribution of image gray levels visually and mathematically, and at the same time show the relationship between each pixel. Except for the Entropy feature used in the present invention and several features described in the figure, other features of GLCM also contain rich information and can represent the global information of the image in different aspects.
[0062] Different features of GLCM have different emphases when describing information in different aspects, and appropriate features or combinations of features need to be selected according to the purpose and specific requirements.
[0063] Step 3: Using the pre-trained neural network, take the texture feature image extracted by the texture feature module as the input, and extract the network output before the SoftMax layer.
[0064] The model used in the present invention is the linear classification model eval_linear in Dino, and the feature output before the SoftMax layer is obtained, with a dimension of 1536. The feature output obtained by the last line of code has a dimension of [N, 1536], where N is the batch_size.
[0065] Specifically, first, input the data into the model to obtain the representation information of the original data in the shallow layer. These features may be edges, corners, textures, etc., which together constitute the initial representation of the input data. Subsequently, these features are reduced in dimension and abstracted through the pooling layer, reducing the spatial size of the data by selecting the maximum value, average value or other statistics within the selected area, while retaining key information as much as possible. This step can reduce the computational complexity of the model and improve the robustness of the model to transformations such as translation and rotation of the input data.
[0066] Through the depth data extracted by the previous layer, further deep information is extracted through the fully connected layer to form a more abstract representation. These representations have gone beyond the intuitive features of the original data, but are a deep understanding of the internal attributes and relationships of the input data. Finally, these features that have undergone multiple abstractions and learning are input into the SoftMax layer. The SoftMax layer converts the deep understanding of the input data by the model into specific classification or regression results by calculating the probability distribution of each category. These results not only reflect the degree of understanding of the input data by the model, but also determine the performance of the model on specific tasks.
[0067] Step 4: Using the feature output extracted by the depth feature extraction module, obtain the optimal weight for feature linear fusion through grid search, and linearly combine the feature outputs.
[0068] The feature fusion module improves the classification accuracy by fusing the feature vectors obtained in the depth feature extraction module and applying them to both the sample and the dataset to be classified simultaneously. The selection of the sample fusion weight is through grid search, and the weight ratio with the best effect is selected from among many weights.
[0069] In this step of feature linear fusion, by selecting appropriate weights, the features are linearly fused. The input of the module is the feature output of the depth adjustment extraction module, and the output of the module is the fused feature vector. The process is shown in Figure 4 .
[0070] In machine learning, grid search is one of the effective methods for searching for appropriate parameters and has good application effects. In machine learning, parameters are manually set before the weight parameters. In the module of this article, it refers to the weight theta of the feature vector of the feature fusion module.
[0071] The core strategy of grid search lies in exhaustive enumeration. Through a predetermined step size or search range, a possible combination of parameter candidates is drawn up, covering all possible combinations of the parameters to be searched. Grid search will apply the generated combinations to the model and adjust them according to the positive feedback of the model's running effect. According to the performance or accuracy on the validation set, it determines which parameter combinations are most beneficial to the generalization of the model.
[0072] In the feature fusion module, grid search can optimize the performance of the model by adjusting the weight (theta) of the feature vector. By continuously trying different weight combinations, those weight representations that can best reflect the features can be found, improving the original classification performance. At the same time, grid search can also help understand the impact of different features on the model performance and provide valuable guidance for subsequent feature selection and feature engineering.
[0073] The weight of the present invention searches for the best weight combination through the above grid search on multiple data sets, using the classification accuracy as a reference standard, and applies the above best weight combination to the classification of the remaining data sets.
[0074] Step 5: Use the feature output after the linear fusion of the test set and the data set to be classified, and use KNN as the classification basis, and evaluate the classification performance according to the consistency between the predicted category and the original category.
[0075] The classification basis of the classification module is KNN. The Euclidean distance is calculated successively between the feature vector of the sample to be classified and the feature vectors of each type of cell sample, and the sum of the distances to a certain cell sample is calculated. The smaller the sum, the more similar it is, and classification is performed according to the size of the sum.
[0076] Step 6: Adopt a PyQt5 graphical interface, which can perform manual operations on this graphical interface and display relevant classification results.
[0077] The system can import the data set of the texture feature images that have been extracted, select the number of samples, and classify the data set based on single feature and multi-feature fusion after clicking Start Classification. The original types and predicted types of different samples and each image can be viewed, and whether the classification is accurate can be viewed through correct / incorrect viewing.
[0078] By clicking on the image name in the table, the image will be displayed in the cell image frame on the left. The number of samples can be modified through the data frame on the left of Start Classification. The classification accuracies based on LBP, GLCM, and feature fusion can be viewed in the image classification situation.
[0079] To verify the role of texture features and the improvement of classification performance by feature fusion, the following control implementation was carried out. Three datasets, namely Centromere, Speckled, and Cytoplasmatic, were selected. The processing method in this experiment was based on the small-sample texture feature fusion classification system designed in the present invention. Without performing feature fusion on the extracted texture features, the feature vectors after deep feature extraction were input into the classification module, and then classification was performed. The classification accuracy is shown in Table 2.
[0080] Table 2 Comparison of the three datasets
[0081]
[0082] From the above results, it can be seen that the classification accuracy of fusing LBP and GLCM texture features is significantly improved compared with that of single features.
Claims
1. A small-sample cell image classification method based on texture feature extraction and fusion, characterized in that It includes the following steps: (1) Preprocess the cell image dataset; perform normalization processing, including size adjustment, random cropping, channel conversion, and normalization operations, to generate input images with a unified specification; (2) Use the LBP algorithm and the GLCM algorithm to extract the texture features of the preprocessed images respectively, generating LBP texture images and GLCM texture images; (3) Adopt a pre-trained neural network model to perform feature encoding on the LBP texture images and GLCM texture images obtained in step (2) respectively, and output deep feature vectors with a dimension of 1536; (4) Determine the optimal weights through grid search, and perform linear weighted fusion on the deep feature vectors of LBP and GLCM to generate fused feature vectors; (5) Based on the fused feature vectors, calculate the Euclidean distance between the image to be classified and the sample images through the KNN algorithm to complete the determination of the cell image category; (6) Display the classification results through a graphical interface, supporting the adjustment of the sample quantity and the visualization of the classification accuracy.
2. The small-sample cell image classification method based on texture feature extraction and fusion according to claim 1, wherein In step (1), the preprocessing includes: size adjustment, random cropping, channel conversion, and normalization operations to generate input images with a unified specification.
3. The small-sample cell image classification method based on texture feature extraction and fusion according to claim 1, wherein In step (2), the implementation method of the LBP algorithm is: calculate the gray-scale contrast between the central pixel and the surrounding pixels in a 3×3 pixel neighborhood, generate an eight-bit binary code and convert it into a decimal value. The formula is: Among them, g c represents the pixel value of the central pixel point, and g p represents the pixel value of the neighboring pixel points.
4. The small-sample cell image classification method based on texture feature extraction and fusion according to claim 1, wherein In step (2), the GLCM algorithm extracts the Entropy feature. The calculation formula is: P(i,j|d,θ)={(x,y)|f(x,y)=i,f(x+dx,y+dx)=j;x,y=0,1,…N-1} where d represents the relative position of the pixel quantity; θ generally considers four directions 0°, 45°, 90°, 135°; i,j=0,1,2,…,,L-1; (x,y) is the pixel coordinate in the image, and L is the number of gray levels of the image.
5. A small-sample cell image classification method based on texture feature extraction and fusion according to claim 1, characterized in that, In step (3), adopt the pre-trained network of the Dino model to output a 1536-dimensional feature vector before the SoftMax layer.
6. The small-sample cell image classification method based on texture feature extraction and fusion according to claim 1, wherein, In step (4), the specific process of grid search is as follows: through a predetermined step size or search range, draw up candidate combinations of parameters. Among them, the candidate combinations cover all combinations of the parameters to be searched; apply the generated combinations to the model and optimize the performance of the model by adjusting the weights of the feature vectors.
7. A small-sample cell image classification method based on texture feature extraction and fusion according to claim 1, characterized in that, In step (5), the determination method of KNN classification is: calculate the sum of the Euclidean distances between the image to be classified and each class of samples, and select the class with the smallest sum as the prediction result.
8. The method for classifying small-sample cell images based on texture feature extraction and fusion according to claim 1, characterized in that In step (6), the graphical interface is implemented based on the PyQt5 framework.
9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the computer program is loaded into the processor, it implements a small-sample cell image classification method based on texture feature extraction and fusion according to any one of claims 1-8.
10. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a small-sample cell image classification method based on texture feature extraction and fusion according to any one of claims 1-8.