Image classification method and device, computer device, storage medium and program product

Abstract

This invention relates to the field of image processing technology, and discloses an image classification method, apparatus, computer device, storage medium, and program product. The method includes: acquiring an original cell image; segmenting the image to obtain multiple image blocks; extracting a first image feature from the original cell image and a second image feature from the multiple image blocks; determining a first information weight and a first non-information weight corresponding to the first image feature to create a first target feature; determining a second information weight and a second non-information weight corresponding to the second image feature to create a second target feature; merging the first and second target features, and performing spatial reconstruction on the merged features to obtain spatially reconstructed features; training a classifier based on the spatially reconstructed features; and inputting the target cell image into the trained classifier to obtain a classification result. By implementing the technical solution of this invention, accurate and effective cell image classification can be achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image classification method and device, computer equipment, a storage medium and a program product. BACKGROUND

[0002] In the image classification task of the deep learning model, feature extraction is performed by fusing a graph convolutional network (GCN) and a convolutional neural network (CNN), aiming to improve the classification performance by using prior knowledge. GCN can effectively transmit information between nodes and learn global features and topological relationships, but its parameters are limited, and its generalization ability is limited, especially in different tasks. Replacing the deep learning model with a texture information branch may lead to too much human intervention, affecting the classification results and the completion of multi-level tasks.

[0003] In order to overcome the limitations of GCN and maintain the ability to extract spatial relationships and global dependencies, a visual neural network architecture (Vision Transformer, ViT) is introduced. On large-scale image datasets such as ImageNet, ViT has shown excellent classification performance. However, the Transformer lacks the inductive bias of CNN and is highly dependent on data, especially in fields such as medical cell images where data is limited, which is prone to overfitting risk. SUMMARY

[0004] Therefore, the present application provides an image classification method and device, computer equipment, a storage medium and a program product to solve the problem of how to effectively use limited data to train a ViT model and improve its generalization ability in the case of data scarcity or high cost.

[0005] In a first aspect, the present application provides an image classification method, comprising: obtaining an original cell image; and segmenting the original cell image to obtain a plurality of image blocks; extracting a first image feature corresponding to the original cell image and a second image feature corresponding to the plurality of image blocks; performing weight separation processing on the first image feature and the second image feature to obtain a first information weight and a first non-information weight corresponding to the first image feature, and a second information weight and a second non-information weight corresponding to the second image feature; wherein the information weight is used to represent the target spatial content part contained in the image feature, and the non-information weight is used to represent the part without target spatial content or the part with target spatial content less than a preset value; creating a first target feature using the first image feature, the first information weight and the first non-information weight; and creating a second target feature using the second image feature, the second information weight and the second non-information weight; merging the first target feature and the second target feature to obtain a merged feature; and performing spatial reconstruction on the merged feature to obtain a spatial reconstruction feature; training a classifier according to the spatial reconstruction feature; and obtaining a target cell image, inputting the target cell image into the trained classifier to obtain a classification result corresponding to the target cell image.

[0006] The image classification method provided by the embodiments of the present application can extract a plurality of image blocks by obtaining an original cell image and segmenting it, which is helpful for processing large-scale data and improving processing efficiency. Extracting image features from the original cell image and its segmented image blocks can capture visual information of different scales and levels, which is conducive to subsequent analysis and processing. Weight separation of image features can separate information weights and non-information weights, which can help reduce the influence of noise and redundant information, improve the expression ability of features and the accuracy of classification. Creating target features using information weights and non-information weights and merging target features from different sources can comprehensively utilize information of different levels and types to improve the effect of classification and recognition. Training a classifier through spatial reconstruction features can fully utilize the spatial information of images to improve the generalization ability and accuracy of the classifier for target cell images. Applying the trained classifier to target cell images can realize the rapid classification and recognition of unknown cell image data, which can effectively improve the precision and efficiency of cell image classification and recognition tasks.

[0007] In one optional implementation, weight separation processing is performed on the first image features to obtain first information weights and first non-information weights corresponding to the first image features, including: normalizing the first image features to obtain corresponding first standard features; weighting the first standard features to obtain corresponding first standard weights; mapping the first standard weights to a preset interval through a preset function to obtain corresponding first mapping weights; obtaining a preset threshold; and determining the weights in the first mapping weights that are higher than the preset threshold as first information weights, and determining the weights in the first mapping weights that are lower than the preset threshold as first non-information weights.

[0008] The image classification method provided in this invention, through normalization processing, ensures that the first image features have a uniform scale and range, which facilitates subsequent weighting and mapping processes, improving the consistency and comparability of feature processing. Weighting the normalized features allows for adjustments based on their importance, strengthening or weakening the influence of specific features, thus improving their expressive power and discriminative power. Mapping the weighted feature weights to a preset interval makes the weights interpretable and standardized, facilitating subsequent thresholding and information extraction steps. Setting a preset threshold and dividing the weights into information weights and non-information weights based on this threshold effectively filters out feature portions with significant information content, reduces interference from useless information, and improves the effectiveness of features and the accuracy of the classifier.

[0009] In one optional implementation, the first target feature includes a first information feature and a first non-information feature; creating the first target feature using the first image feature, the first information weight, and the first non-information weight includes: multiplying the first image feature by the first information weight to obtain the first information feature, and multiplying the first image feature by the first non-information weight to obtain the first non-information feature.

[0010] The image classification method provided in this invention, by multiplying by information weights, can strengthen and highlight the information-rich parts of image features, giving these parts more weight and influence in the final target features. This helps to more accurately identify and classify target cell images during subsequent classifier training. By multiplying by non-information weights, the influence of potentially noisy or irrelevant parts of image features can be reduced or suppressed. This helps to improve the cleanliness and purity of features, making the final target features more accurately reflect the true features and patterns in the image. Integrating information weights and non-information weights can fully utilize the differences between them, thereby more accurately representing various aspects of image features. This comprehensive utilization helps to optimize feature quality and classifier performance, making the overall image analysis and classification task more efficient and reliable.

[0011] In one optional implementation, extracting corresponding second image features from multiple image blocks includes: obtaining position encoding vectors corresponding to multiple image blocks, and performing linear projection on the multiple image blocks to obtain projection vectors corresponding to multiple image blocks; adding the projection vectors and position encoding vectors to obtain second image features corresponding to multiple image blocks.

[0012] The image classification method provided in this invention effectively embeds the location information of each image patch into the feature vector by introducing a location encoding vector. This helps the model distinguish the spatial locations of image patches when processing them, thereby more accurately understanding the structure and content of the image. Through linear projection, the original features of each image patch can be mapped to a higher-dimensional space. This helps increase the diversity and expressive power of the features, thus better capturing key information and patterns in the image patches. By adding the projection vector and the location encoding vector, feature fusion is achieved. This fusion effectively combines spatial location information and the content features of the image patches, thereby generating richer and more meaningful second image features. The feature vector after integrating the location encoding not only increases the model's ability to understand image content and structure but also improves the model's generalization ability and classification accuracy in complex scenes.

[0013] In one optional implementation, the second image features are subjected to weight separation processing to obtain second information weights and second non-information weights corresponding to the second image features. This includes: processing the second image features according to a multi-head self-attention mechanism to obtain a multi-head feature vector; wherein the multi-head feature vector is used to characterize the information of the second image features at different points of interest; normalizing the multi-head feature vector to obtain corresponding second standard features; weighting the second standard features to obtain corresponding second standard weights; mapping the second standard weights to a preset interval through a preset function to obtain corresponding second mapping weights; determining the weights in the second mapping weights that are higher than a preset threshold as second information weights, and determining the weights in the second mapping weights that are lower than the preset threshold as second non-information weights.

[0014] The image classification method provided in this invention employs a multi-head self-attention mechanism that simultaneously focuses on the importance of different locations within image features. This mechanism effectively captures the complex relationships between different parts of the image by calculating the attention weights between each location and other locations. The multi-head feature vector can characterize the information of the second image features at different points of interest, meaning it can treat image features from multiple perspectives, thus capturing the image's content and contextual information more comprehensively. Normalizing the multi-head feature vector helps maintain its stability and comparability. Scaling the feature vectors to a similar numerical range avoids training instability or convergence difficulties caused by different numerical ranges. Weighted adjustments to the second standard features can strengthen or weaken the influence of specific parts according to different tasks or needs. The mapping process of the preset function further optimizes the weight distribution, making it more suitable for specific application needs and goals. By setting a threshold, the second mapping weights can be divided into informational weights and non-informational weights. This helps the model more effectively filter and utilize key information in subsequent processing, thereby improving the accuracy and efficiency of the task.

[0015] In one optional implementation, the second image features are processed according to a multi-head self-attention mechanism to obtain multi-head feature vectors, including: obtaining a pre-set query weight matrix, key weight matrix, and value weight matrix; multiplying the second image features by the query weight matrix, key weight matrix, and value weight matrix respectively to obtain corresponding query vectors, key vectors, and value vectors; calculating the dot product of the query vector and the key vector, and determining the dot product as the attention score; performing a weighted summation of the attention scores to obtain single-head feature vectors corresponding to each attention head; merging the single-head feature vectors, and performing a linear transformation on the merged feature vectors to obtain multi-head feature vectors corresponding to multiple attention heads.

[0016] The image classification method provided in this invention utilizes a self-attention mechanism that allows the model to compute the query, key, and value vectors at each location in parallel. This enables the simultaneous processing of information from various parts of the image, regardless of their positional order. This global attention capability helps the model understand image content without being affected by positional distance, thereby improving the global consistency and accuracy of feature extraction. By calculating the dot product of the query and key vectors and applying a weighted sum, the attention weights can be dynamically adjusted according to the importance of each location. This flexibility allows the model to focus more effectively on key information when dealing with complex image scenes, improving the accuracy and robustness of feature extraction. Merging and linearly transforming the feature vectors generated by a single attention head yields multi-head feature vectors. These multi-head feature vectors encompass different points of interest and perspectives, enabling the model to comprehensively utilize multiple attention heads for a deeper and more comprehensive analysis of image features. Through the multi-head self-attention mechanism, the model can more effectively extract and represent complex relationships and structures in image features. This enhanced feature representation capability helps improve the model's performance in visual tasks. Self-attention mechanisms have a certain degree of anti-interference ability when processing image features. They can effectively deal with interference and noise in local areas of the image, thereby improving the robustness and generalization ability of the model in complex backgrounds.

[0017] In one optional implementation, the second target feature includes a second information feature and a second non-information feature; creating the second target feature using the second image feature, the second information weight, and the second non-information weight includes: multiplying the second image feature by the second information weight to obtain the second information feature, and multiplying the second image feature by the second non-information weight to obtain the second non-information feature.

[0018] The image classification method provided in this invention effectively filters and distinguishes important and unimportant information in an image by multiplying a second image feature by information weights and non-information weights. This operation helps the model focus on processing the most meaningful information in the current cell image, thereby improving the model's efficiency and performance. Distinguishing between the second information feature and the second non-information feature allows the model to more finely control the amount and quality of information processed. After generating the second information feature and the second non-information feature, the model can more effectively adapt to specific task requirements. By effectively processing these two types of information, the model's performance and generalization ability in various visual tasks can be significantly improved. Through the separation of information features and non-information features, the model can be more efficient in processing large-scale image data. This optimization helps reduce the model's computational burden and resource consumption, improving overall processing speed and real-time performance.

[0019] In one optional implementation, merging the first target feature and the second target feature to obtain a merged feature includes: fusing the first information feature and the second information feature using a convolutional layer to obtain a first fused feature; fusing the first non-information feature and the second non-information feature to obtain a second fused feature; merging the first fused feature and the second fused feature, and performing convolution processing on the merged feature to obtain the merged feature.

[0020] The image classification method provided in this invention fuses first and second information features, as well as first and second non-information features, through convolutional layers, effectively integrating feature information from different sources and with different properties. This fusion helps to comprehensively utilize the advantages of multiple feature sources, improving the model's ability to express and analyze complex image information. Merging the first and second fused features and then processing them through convolution to obtain merged features allows the model to richly express image features from multiple perspectives. This information complementarity and richness helps reduce information loss and error propagation, improving the model's comprehensive understanding and processing capabilities of image content. Processing the stitched features through convolutional layers effectively extracts and enhances the spatial information of the features. Convolutional operations have local perception capabilities and parameter sharing characteristics, effectively reducing the number of parameters and computational complexity, while improving the accuracy and stability of feature representation.

[0021] In one optional implementation, spatial reconstruction of the merged features to obtain spatially reconstructed features includes: performing weight separation processing on the merged features to obtain a third information weight and a third non-information weight corresponding to the merged features; multiplying the merged features by the third information weights to obtain a third information feature, and multiplying the merged features by the third non-information weights to obtain a third non-information feature; splitting the third information feature into a first information sub-feature and a second information sub-feature, and splitting the third non-information feature into a third information sub-feature and a fourth information sub-feature; adding the first information sub-feature and the fourth information sub-feature element-wise to obtain a first cross feature; adding the second information sub-feature and the third information sub-feature element-wise to obtain a second cross feature; and merging the first cross feature and the second cross feature to obtain the spatially reconstructed features.

[0022] The image classification method provided in this invention, by performing weight separation processing on merged features to generate third information weights and third non-information weights respectively, can finely control the influence of different types of features in subsequent processing. This refined feature weighting helps the model more effectively distinguish and utilize important and non-important information in the image, thereby improving the model's performance and robustness in handling complex visual tasks. Multiplying the merged features by the third information weights and third non-information weights respectively to generate third information features and third non-information features, and further decomposing them into multiple information sub-features and non-information sub-features, can enhance the model's ability to understand different levels and angles of image content, enabling the model to analyze and process input data more comprehensively. By adding the first and fourth information sub-features element-wise, and adding the second and third information sub-features element-wise, first cross features and second cross features are generated. This cross feature generation can capture the interaction and correlation between different information subsets, helping to improve the model's performance and generalization ability in complex scenes. By merging the first and second cross features to generate spatial reconstruction features, information from different sub-features and cross features is integrated, forming a more comprehensive and complete feature representation. This spatial reconstruction feature helps improve the model's understanding of the overall structure and spatial layout of an image, thereby enhancing the model's performance and accuracy in image processing tasks.

[0023] Secondly, the present invention provides an image classification apparatus, comprising: an acquisition module for acquiring an original cell image; and segmenting the original cell image to obtain multiple image blocks; an extraction module for extracting corresponding first image features from the original cell image and corresponding second image features from the multiple image blocks; and a processing module for performing weight separation processing on the first image features and the second image features to obtain a first information weight and a first non-information weight corresponding to the first image features, and a second information weight and a second non-information weight corresponding to the second image features; wherein the information weight is used to characterize the portion of the image features containing target space content, and the non-information weight is used to characterize the image features containing target space content. The system includes a feature module for creating a first target feature using first image features, first information weights, and first non-information weights; a second target feature using second image features, second information weights, and second non-information weights; a merging module for merging the first and second target features to obtain merged features; spatial reconstruction of the merged features to obtain spatially reconstructed features; a classification module for training a classifier based on the spatially reconstructed features; and acquiring a target cell image, inputting the target cell image into the trained classifier to obtain the classification result corresponding to the target cell image.

[0024] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the image classification method described in the first aspect or any corresponding embodiment thereof.

[0025] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the image classification method described in the first aspect or any corresponding embodiment thereof.

[0026] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the image classification method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0027] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating an image classification method according to an embodiment of the present invention;

[0029] Figure 2 This is a flowchart illustrating another image classification method according to an embodiment of the present invention;

[0030] Figure 3 This is a flowchart illustrating another image classification method according to an embodiment of the present invention;

[0031] Figure 4 This is a flowchart illustrating another image classification method according to an embodiment of the present invention;

[0032] Figure 5 This is a structural block diagram of an image classification device according to an embodiment of the present invention;

[0033] Figure 6 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] In network structures that combine GCN and CNN for classification, although the GCN branch effectively transmits information between nodes, enabling the model to learn richer global features and topological relationships between cells, it has fewer learnable parameters, thus limiting its generalization ability across different tasks. On the other hand, directly replacing the original deep learning model with the texture information branch may lead to excessive human intervention, affecting the model's adaptability to classification tasks at different levels.

[0036] To overcome the limitations of GCNs while preserving their ability to extract spatial relationships and global dependencies, the ViT structure has been introduced in related techniques. ViT effectively captures global dependencies in images through a self-attention mechanism, allowing even distant regions to influence each other. This gives ViT an advantage in understanding the overall context of an image. In contrast, while GCNs can handle global relationships between nodes, their ability to directly understand the global context of regular image data may not be as direct as that of ViTs.

[0037] Furthermore, the ViT model is not limited by specific data structures, allowing for more flexible handling of images at different resolutions and multi-scale analysis. The scalability of the Transformer architecture allows for effective performance improvements by increasing model depth or width, while GCN's scalability is more limited due to the characteristics of its graph structure.

[0038] Although ViT demonstrates excellent classification performance on large-scale datasets such as ImageNet, it may face the risk of overfitting in specific domain applications such as small datasets, due to the lack of inductive biases of CNNs, such as locality and translation invariance.

[0039] In view of this, to improve the classification accuracy of the model and achieve balanced classification performance among different categories, the technical solution of this invention first employs a hybrid approach of traditional data augmentation and texture filtering amplification for image preprocessing. Especially in datasets with fewer categories, amplification using various filter sizes and angles is employed to balance the number of samples in each category as much as possible. Secondly, a two-branch classification network is constructed. The self-attention mechanism of the ViT model is used to highlight important cellular regions and global dependencies in the learned image. Through a channel re-representation module, features learned from ViT and the EfficientNet backbone network are fused to enhance feature representation and classification performance. These improvements retain the advantages of EfficientNet as the backbone network while avoiding the challenge of training on large datasets when directly using the ViT model. Simultaneously, by introducing ViT's self-attention mechanism and positional encoding, the learning of spatial information and global dependencies is effectively combined, further improving the model's classification performance. Ultimately, this results in improved overall classification performance and a better balance and classification accuracy among different categories.

[0040] According to an embodiment of the present invention, an image classification method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0041] This embodiment provides an image classification method that can be used in computer devices, such as desktop computers and laptops. Figure 1 This is a flowchart of an image classification method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0042] Step S101: Obtain the original cell image; and segment the original cell image to obtain multiple image blocks.

[0043] Raw cell images are unprocessed images obtained from biological samples, such as those acquired using instruments like microscopes. Specifically, in laboratory or clinical settings, researchers or physicians collect cell samples, such as those obtained from tissue sections or cell cultures. These samples are then imaged using instruments like microscopes for subsequent analysis and research.

[0044] Multiple image patches refer to dividing the entire original cell image into multiple small regions or sub-images. These image patches can appear in the form of squares or rectangles, and each image patch contains local information of the cell image. Specifically, experts or researchers can manually draw boundaries or regions to divide the image into different parts based on cell morphology, color, and other characteristics; alternatively, computer vision techniques and algorithms can be used for automatic segmentation, such as thresholding, edge detection, region growing, and deep learning methods. These methods can automatically segment multiple image patches according to the characteristics and requirements of the original cell image, and no limitation is made here.

[0045] Step S102: Extract the corresponding first image features from the original cell image, and extract the corresponding second image features from multiple image blocks.

[0046] The first image feature refers to a set of features extracted from the entire original cell image to describe the overall content and characteristics of the image, such as the overall color distribution, shape features, and texture information. Specifically, image processing libraries (such as OpenCV), machine learning libraries (such as scikit-learn), or deep learning libraries (such as TensorFlow and PyTorch) can be used to calculate the feature values ​​of the original cell image, and the calculated feature values ​​can be represented in vector or other numerical form for subsequent analysis and processing.

[0047] The second image feature refers to the set of features extracted from each image patch to describe local features. Specifically, for each image patch, local features such as color histograms and texture descriptors are extracted. The local features of each image patch are combined into a vector or feature set to represent the visual information of that image patch. Then, the feature vectors of all image patches are concatenated or aggregated to form the second image feature representation of the entire original cell image.

[0048] Step S103: Perform weight separation processing on the first image feature and the second image feature to obtain the first information weight and the first non-information weight corresponding to the first image feature, and the second information weight and the second non-information weight corresponding to the second image feature; wherein, the information weight is used to characterize that the image feature contains a target space content part, and the non-information weight is used to characterize that the image feature does not contain a target space content part or the proportion of the target space content part is less than a preset value.

[0049] The target spatial content component is used to represent useful and expressive spatial content, such as key parts like the cell nucleus and cytoplasm in a cell image. Understandably, information weights represent the presence of useful target spatial content in image features, while non-information weights represent spatial content with little or no information, or whose proportion is less than a preset value, such as background, noise, irrelevant structures, or low-contrast areas in a cell image. The preset value is a threshold or limiting condition used to determine which spatial content components are considered informational or non-informational.

[0050] The first information weight is used to characterize the spatial content portions of the first image features that are useful and expressive. This first information weight primarily captures important informative features in the first image, helping to distinguish or describe key content in the original cell image. The first non-information weight is used to characterize the spatial content portions of the first image features that are less informative or lack information. This first non-information weight primarily covers noise, redundancy, or irrelevant information in the first image. The second information weight is used to characterize the spatial content portions of the second image features that are useful and expressive. The second non-information weight is used to characterize the spatial content portions of the second image features that are less informative or lack information. Specifically, a model or objective function can be predefined to distinguish between informative and non-informative portions of the first and second image features. Through training or optimization, the model parameters or the weights of the objective function are adjusted to ensure that the information weights and non-information weights are reasonably separated.

[0051] Step S104: Create a first target feature using the first image feature, the first information weight, and the first non-information weight; and create a second target feature using the second image feature, the second information weight, and the second non-information weight.

[0052] The first target feature is a feature obtained by weighting the first image features according to corresponding first information weights and first non-information weights. Specifically, the first information weights are used to weight the important information parts of the first image features, and the first non-information weights are used to weight the non-information parts of the first image features. The weighted important information parts and non-information parts are combined to obtain the first target feature.

[0053] The second target feature is obtained by weighting the second image features according to corresponding second information weights and second non-information weights. Specifically, the second information weights are used to weight the important information parts of the second image features, and the second non-information weights are used to weight the non-information parts of the second image features. The weighted important information parts and non-information parts are combined to obtain the second target feature.

[0054] Step S105: Merge the first target feature and the second target feature to obtain merged features; and perform spatial reconstruction on the merged features to obtain spatially reconstructed features.

[0055] Feature merging refers to combining the first target feature and the second target feature to form a unified feature representation. Specifically, the first target feature and the second target feature can be concatenated into a longer vector in a preset order to obtain the merged feature; alternatively, attention mechanisms, neural network structures, or other methods can be used to merge two feature maps into a new feature map, which is not limited here.

[0056] Spatial reconstruction features refer to the process of reconstructing abstract features back to the space of the original data, such as the original image space corresponding to the original cell image, based on merged features. Specifically, merged features are used as input and processed by selected spatial reconstruction techniques, such as decoder networks, deconvolution, or unpooling, to generate spatial reconstruction features that restore the structure and features of the original cell image data as much as possible.

[0057] Step S106: Train the classifier based on spatial reconstruction features; and acquire the target cell image, input the target cell image into the trained classifier, and obtain the classification result corresponding to the target cell image.

[0058] Before training the classifier based on spatial reconstruction features, it is necessary to obtain the correct classification label for each original cell image sample. These labels can be manually labeled or obtained through other methods. Specifically, all the spatial reconstruction features and their corresponding labels obtained above are used to form a training set, where each sample consists of a spatial reconstruction feature vector and its corresponding classification label. A suitable classifier model for the task is selected, such as a Support Vector Machine (SVM), a Deep Neural Network (DNN), or a Random Forest. The prepared training set is then input into the selected classifier model for training. During training, the classifier model adjusts its internal parameters based on the spatial reconstruction features and corresponding labels to correctly classify unseen data. Furthermore, after training, a validation set can be used to evaluate the model to assess its classification accuracy, generalization ability, and other metrics. This helps determine whether the model can effectively handle new data and checks for overfitting or underfitting issues.

[0059] Target cell images refer to cell image samples to be classified; these images can be new and previously unseen data. Specifically, target cell images of unknown classification are obtained from real-world application scenarios and transformed into a form suitable for classifier input. This involves converting the target cell images into a feature representation similar to the spatial reconstruction features used during training. The transformed features are then input into the trained classifier model. Based on its trained model parameters, the classifier predicts the category to which the target cell image belongs. The classification result, i.e., the category label or classification probability of the target cell image, is then obtained from the classifier's output.

[0060] The image classification method provided in this invention, by acquiring and segmenting the original cell image, can extract multiple image patches, which helps in processing large-scale data and improves processing efficiency. Extracting image features from the original cell image and its segmented image patches captures visual information at different scales and levels, which is beneficial for subsequent analysis and processing. Weighting the image features separately, processing information weights and non-information weights, helps reduce the influence of noise and redundant information, improving the expressive power of features and the accuracy of classification. Creating target features using information weights and non-information weights, and merging target features from different sources, can comprehensively utilize information at different levels and types, improving the classification and recognition effect. Training the classifier through spatial reconstruction features can fully utilize the spatial information of the image, improving the classifier's generalization ability and accuracy for target cell images. Applying the trained classifier to target cell images enables rapid classification and recognition of unknown cell image data, effectively improving the accuracy and efficiency of cell image classification and recognition tasks.

[0061] This embodiment provides an image classification method that can be used in computer devices, such as desktop computers and laptops. Figure 2 This is a flowchart of an image classification method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0062] Step S201: Obtain the original cell image; and segment the original cell image to obtain multiple image patches. For details, please refer to [link to relevant documentation]. Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0063] Step S202 involves extracting the corresponding first image features from the original cell image and extracting the corresponding second image features from multiple image patches. For details, please refer to [link to relevant documentation]. Figure 1 Step S102 of the illustrated embodiment will not be described again here.

[0064] Step S203: Perform weight separation processing on the first image feature and the second image feature to obtain the first information weight and the first non-information weight corresponding to the first image feature, and the second information weight and the second non-information weight corresponding to the second image feature; wherein, the information weight is used to characterize the spatial content part of the image feature that is useful and expressive, and the non-information weight is used to characterize the spatial content part of the image feature that is less or no information.

[0065] Specifically, step S203 includes:

[0066] Step S2031: Normalize the first image features to obtain the corresponding first standard features.

[0067] The first standard feature is the feature obtained after normalizing the first image feature. Specifically, the first image feature is normalized using a group normalization (GN) layer to obtain the first standard feature after group normalization and weighting.

[0068] For example, the above normalization process can be achieved through... Let F be the first image feature, GN(F) be the group normalization operation performed on the first image feature, μ be the mean of the first image feature, σ be the standard deviation of the first image feature, ε be a small constant used to avoid division by zero, and α and β be trainable weight parameters used to adjust the normalized features. OUT It is the first standard feature.

[0069] Step S2032: Weight the first standard features to obtain the corresponding first standard weights.

[0070] The first standard weight is used to characterize the importance or information content of each position in the first standard feature. Specifically, the first standard feature is weighted using a pre-established formula W = Gate(Sigmoid(Wα(GN(x)))), where Wα = Gate(Sigmoid(Wα(GN(x)))) OUT )))) is achieved through the normalization operation GN(F) OUT The first standard weights are calculated.

[0071] Step S2033: Map the first standard weight to a preset interval through a preset function to obtain the corresponding first mapped weight.

[0072] Preset functions are used to adjust the numerical range and distribution of the first standard weights to suit the specific algorithm or task requirements. For example, the sigmoid function can be used to restrict the first standard weights within a preset range to improve the stability and efficiency of the algorithm.

[0073] The preset interval determines the numerical range of the first standard weight, for example, it can be the range of (0, 1). A suitable preset interval can ensure that the value of the first standard weight is within the processable range and avoid numerical overflow or instability.

[0074] The first mapped weight is the result of processing the first standard weights using a preset function. Specifically, the first standard weights are mapped to a preset interval using the Sigmoid function or other designed functions to obtain the first mapped weight, which represents the weight value after standardization and mapping.

[0075] Step S2034: Obtain a preset threshold; and determine the weights in the first mapping weights that are higher than the preset threshold as first information weights, and determine the weights in the first mapping weights that are lower than the preset threshold as first non-information weights.

[0076] The preset threshold is derived from previous experiments or hyperparameter adjustments. This threshold is used to distinguish which weights are considered useful information weights and which are considered useless redundant weights. Specifically, the first mapping weights are classified according to the preset threshold.

[0077] The first information weight is the portion of the first mapping weight that exceeds a preset threshold; that is, the weight containing useful and expressive spatial content. The first non-information weight is the portion of the first mapping weight that falls below the preset threshold; that is, the weight containing little or no information and spatial redundancy. Specifically, the gating function constrains the first mapping weight within a preset interval based on the output of the Sigmoid function. The gating function can be simply represented as a threshold function, such as the Gate function, which sets the portion of the first mapping weight greater than the preset threshold to 1, forming the first information weight; and sets the portion less than the preset threshold to 0, forming the first non-information weight.

[0078] Step S2035: Perform weight separation processing on the second image features to obtain the second information weights and second non-information weights corresponding to the second image features. For details, please refer to [link to relevant documentation]. Figure 1 Step S103 of the illustrated embodiment will not be described again here.

[0079] The image classification method provided in this invention, through normalization processing, ensures that the first image features have a uniform scale and range, which facilitates subsequent weighting and mapping processes, improving the consistency and comparability of feature processing. Weighting the normalized features allows for adjustments based on their importance, strengthening or weakening the influence of specific features, thus improving their expressive power and discriminative power. Mapping the weighted feature weights to a preset interval makes the weights interpretable and standardized, facilitating subsequent thresholding and information extraction steps. Setting a preset threshold and dividing the weights into information weights and non-information weights based on this threshold effectively filters out feature portions with significant information content, reduces interference from useless information, and improves the effectiveness of features and the accuracy of the classifier.

[0080] Step S204: Create a first target feature using the first image feature, the first information weight, and the first non-information weight; and create a second target feature using the second image feature, the second information weight, and the second non-information weight.

[0081] Specifically, step S204 includes:

[0082] Step S2041: Multiply the first image feature by the first information weight to obtain the first information feature, and multiply the first image feature by the first non-information weight to obtain the first non-information feature.

[0083] The first target feature includes a first information feature and a first non-information feature. Specifically, the first image feature is multiplied by a first information weight, and the informational portion of the first image feature selected by the first information weight is extracted. The first image feature is multiplied by a first non-information weight, and the non-information portion of the first image feature selected by the first non-information weight is extracted.

[0084] Step S2042: Create second target features using the second image features, second information weights, and second non-information weights. See details below. Figure 1 Step S104 of the illustrated embodiment will not be described again here.

[0085] Step S205 involves merging the first target feature and the second target feature to obtain merged features; and then performing spatial reconstruction on the merged features to obtain spatially reconstructed features. For details, please refer to [link to details]. Figure 1 Step S105 of the illustrated embodiment will not be described again here.

[0086] Step S206 involves training the classifier based on spatial reconstruction features; acquiring the target cell image, inputting the target cell image into the trained classifier, and obtaining the classification result corresponding to the target cell image. For details, please refer to [link to relevant documentation]. Figure 1 Step S106 of the illustrated embodiment will not be described again here.

[0087] The image classification method provided in this invention, by multiplying by information weights, can strengthen and highlight the information-rich parts of image features, giving these parts more weight and influence in the final target features. This helps to more accurately identify and classify target cell images during subsequent classifier training. By multiplying by non-information weights, the influence of potentially noisy or irrelevant parts of image features can be reduced or suppressed. This helps to improve the cleanliness and purity of features, making the final target features more accurately reflect the true features and patterns in the image. Integrating information weights and non-information weights can fully utilize the differences between them, thereby more accurately representing various aspects of image features. This comprehensive utilization helps to optimize feature quality and classifier performance, making the overall image analysis and classification task more efficient and reliable.

[0088] This embodiment provides an image classification method that can be used in computer devices, such as desktop computers and laptops. Figure 3 This is a flowchart of an image classification method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0089] Step S301: Acquire the original cell image; and segment the original cell image to obtain multiple image patches. For details, please refer to [link to relevant documentation]. Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0090] Step S302: Extract the corresponding first image features from the original cell image, and extract the corresponding second image features from multiple image blocks.

[0091] Specifically, step S302 includes:

[0092] Step S3021: Extract the corresponding first image features from the original cell image. See details below. Figure 1 Step S102 of the illustrated embodiment will not be described again here.

[0093] Step S3022: Obtain the position encoding vectors corresponding to multiple image blocks, and perform linear projection on the multiple image blocks to obtain the projection vectors corresponding to the multiple image blocks.

[0094] The positional encoding vector represents the relative position of each image patch within the entire image, with each image patch corresponding to a single positional encoding vector. Specifically, the positional encoding vector can be predefined or learned through a model.

[0095] Linear projection of the features of each image patch can be understood as mapping the feature vectors of each image patch to a higher-dimensional or lower-dimensional vector space. Specifically, linear projection can be achieved using matrix multiplication to obtain projection vectors corresponding to multiple image patches, where the projection matrix is ​​a learnable parameter used to adjust the representation of image patch features.

[0096] Step S3023: Add the projection vector and the position encoding vector together to obtain the second image features corresponding to multiple image blocks.

[0097] The projection vectors of each image patch obtained above are added element-wise to their corresponding positional encoding vectors. The spatial information (via the positional encoding vector) and feature information (via the projection vector) of each image patch are integrated to obtain the second image features corresponding to multiple image patches, thereby enhancing the model's ability to represent image patch features. For example, this can be achieved using formula z. i =x i +x i *σ(DWConv(x i Let z be represented as z. i It is the second image feature, x i This represents the vector representation of each image patch after transformation through a linear projection layer, i = 1, 2, ..., N, DWConv(x i () is a depthwise separable convolution operation that performs channel-wise convolution on x. i Perform spatial convolution operations.

[0098] The image classification method provided in this invention effectively embeds the location information of each image patch into the feature vector by introducing a location encoding vector. This helps the model distinguish the spatial locations of image patches when processing them, thereby more accurately understanding the structure and content of the image. Through linear projection, the original features of each image patch can be mapped to a higher-dimensional space. This helps increase the diversity and expressive power of the features, thus better capturing key information and patterns in the image patches. By adding the projection vector and the location encoding vector, feature fusion is achieved. This fusion effectively combines spatial location information and the content features of the image patches, thereby generating richer and more meaningful second image features. The feature vector after integrating the location encoding not only increases the model's ability to understand image content and structure but also improves the model's generalization ability and classification accuracy in complex scenes.

[0099] Step S303: Perform weight separation processing on the first image feature and the second image feature to obtain the first information weight and the first non-information weight corresponding to the first image feature, and the second information weight and the second non-information weight corresponding to the second image feature; wherein, the information weight is used to characterize the spatial content part of the image feature that is useful and expressive, and the non-information weight is used to characterize the spatial content part of the image feature that is less or no information.

[0100] Specifically, step S303 includes:

[0101] Step S3031: Perform weight separation processing on the first image features to obtain the first information weight and the first non-information weight corresponding to the first image features. For details, please refer to [link to relevant documentation]. Figure 1 Step S103 of the illustrated embodiment will not be described again here.

[0102] Step S3032: Process the second image features according to the multi-head self-attention mechanism to obtain the multi-head feature vector; wherein, the multi-head feature vector is used to represent the information of the second image features at different points of interest.

[0103] Multi-head feature vectors contain relational information learned in different spaces by different attention heads. Each head can focus on different points of interest, such as local features, global features, or features at different scales. Specifically, the input second image features are processed through queries, keys, and values. The importance of each location or region is calculated using a weighted attention mechanism. Finally, the outputs of multiple heads are combined to form a multi-head feature vector, which represents the information of the image at different points of interest.

[0104] In some optional implementations, step S3032 above includes:

[0105] Step a1: Obtain the pre-set query weight matrix, key weight matrix, and value weight matrix.

[0106] The query weight matrix maps the input second image features to a query vector (Q), the key weight matrix maps the input second image features to a key vector (K), and the value weight matrix maps the input second image features to a value vector (V). Specifically, three predefined weight matrices are obtained to map the second image features to the vector spaces of query, key, and value.

[0107] Step a2: Multiply the second image features by the query weight matrix, key weight matrix, and value weight matrix respectively to obtain the corresponding query vector, key vector, and value vector.

[0108] The second image features are multiplied by a predefined query weight matrix, key weight matrix, and value weight matrix to generate corresponding query vectors, key vectors, and value vectors, which will be used for subsequent attention mechanism calculations.

[0109] Step a3: Calculate the dot product of the query vector and the key vector, and determine the dot product as the attention score.

[0110] Attention scores are used to characterize the similarity or relevance between query vectors and key vectors. Specifically, for each attention head h, the dot product of the query vector and key vector is calculated, which can be expressed as Attention Score = QK. T The attention score determines how much attention is given to each part of the input sequence when generating each output element.

[0111] Step a4: Perform a weighted summation of the attention scores to obtain the single-head feature vector corresponding to each attention head.

[0112] A single-head feature vector is used to characterize the integrated representation of the input sequence (attention score) by each attention head given a query vector. Each attention head h computes an attention weight vector, which is used to perform a weighted summation of the value vectors to obtain the single-head feature vector head. i Specifically, for each head, the attention weight vector is calculated. The attention score can be normalized using the softmax function, and the attention weights are used to perform a weighted sum of the value vectors to obtain the single-head feature vector. i head i =Attention(QW i Q W i K VW i V ), where W i Q W i K W i V It is the weight matrix corresponding to each head.

[0113] Step a5: Merge the single-head feature vectors and perform a linear transformation on the merged feature vectors to obtain multi-head feature vectors corresponding to multiple attention heads.

[0114] The single-head feature vectors of each attention head are concatenated to obtain a multi-head feature vector. A linear transformation is then used to convert the multi-head feature vector into the final multi-head feature vector. Specifically, this can be achieved using MultiHead(Q, K, V) = Concat(head1, head2, ..., headh)W.O To represent, W O It is the weight matrix for the output linear transformation.

[0115] In the above implementation, the self-attention mechanism allows the model to compute the query, key, and value vectors at each location in parallel, thus processing information from various parts of the image simultaneously, regardless of positional order. This global attention capability helps the model understand image content without being affected by positional distance, thereby improving the global consistency and accuracy of feature extraction. By computing the dot product of the query and key vectors and applying a weighted sum, the attention weights can be dynamically adjusted according to the importance of each location. This flexibility enables the model to focus more effectively on key information when dealing with complex image scenes, improving the accuracy and robustness of feature extraction. Merging and linearly transforming the feature vectors generated by a single attention head yields multi-head feature vectors. These multi-head feature vectors cover different points of interest and perspectives, allowing the model to comprehensively utilize multiple attention heads for a deeper and more comprehensive analysis of image features. Through the multi-head self-attention mechanism, the model can more effectively extract and express complex relationships and structures in image features. This enhanced feature representation capability helps improve the model's performance in visual tasks. The self-attention mechanism has a certain degree of anti-interference capability when processing image features, effectively dealing with interference and noise in local areas of the image, thereby improving the model's robustness and generalization ability in complex backgrounds.

[0116] Step S3033: Normalize the multi-head feature vector to obtain the corresponding second standard feature.

[0117] The second standard feature is the feature obtained after normalizing the multi-head feature vector. Specifically, the multi-head feature vector is normalized using a group normalization (GN) layer to obtain the second standard feature after group normalization and weighting.

[0118] Step S3034: Weight the second standard features to obtain the corresponding second standard weights.

[0119] The second standard weight is used to characterize the importance or information content of each position in the second standard feature. Specifically, it utilizes a pre-established formula. The second standard features are weighted, where the second standard weights are calculated through a normalization operation.

[0120] Step S3035: Map the second standard weights to a preset interval using a preset function to obtain the corresponding second mapped weights.

[0121] The second mapped weights are the result of processing the second standard weights using a preset function. Specifically, the second standard weights are mapped to a preset interval using the Sigmoid function or other designed functions to obtain the second mapped weights, which represent the weight values ​​after standardization and mapping.

[0122] Step S3036: The weights in the second mapping weights that are higher than a preset threshold are determined as the second information weights, and the weights in the second mapping weights that are lower than the preset threshold are determined as the second non-information weights.

[0123] The second information weight is the portion of the second mapping weight that exceeds a preset threshold; that is, weights containing useful and expressive spatial content. The second non-information weight is the portion of the second mapping weight that falls below a preset threshold; that is, weights containing little or no information and spatial redundancy. Specifically, the gating function constrains the second mapping weights within a preset range based on the output of the Sigmoid function. The gating function can be simply represented as a threshold function, such as the Gate function, which sets the portion of the second mapping weights above the preset threshold to 1, forming the second information weights; and sets the portion below the preset threshold to 0, forming the second non-information weights.

[0124] The image classification method provided in this invention employs a multi-head self-attention mechanism that simultaneously focuses on the importance of different locations within image features. This mechanism effectively captures the complex relationships between different parts of the image by calculating the attention weights between each location and other locations. The multi-head feature vector can characterize the information of the second image features at different points of interest, meaning it can treat image features from multiple perspectives, thus capturing the image's content and contextual information more comprehensively. Normalizing the multi-head feature vector helps maintain its stability and comparability. Scaling the feature vectors to a similar numerical range avoids training instability or convergence difficulties caused by different numerical ranges. Weighted adjustments to the second standard features can strengthen or weaken the influence of specific parts according to different tasks or needs. The mapping process of the preset function further optimizes the weight distribution, making it more suitable for specific application needs and goals. By setting a threshold, the second mapping weights can be divided into informational weights and non-informational weights. This helps the model more effectively filter and utilize key information in subsequent processing, thereby improving the accuracy and efficiency of the task.

[0125] Step S304: Create a first target feature using the first image feature, the first information weight, and the first non-information weight; and create a second target feature using the second image feature, the second information weight, and the second non-information weight.

[0126] Specifically, step S304 includes:

[0127] Step S3041: Create a first target feature using the first image features, the first information weights, and the first non-information weights. For details, please refer to [link to relevant documentation]. Figure 1 Step S104 of the illustrated embodiment will not be described again here.

[0128] Step S3042: Multiply the second image feature by the second information weight to obtain the second information feature, and multiply the second image feature by the second non-information weight to obtain the second non-information feature.

[0129] The second target feature includes a second information feature and a second non-information feature. Specifically, the second image feature is multiplied by the second information weight to extract the informational portion of the second image feature selected by the second information weight. The second image feature is multiplied by the second non-information weight to extract the non-information portion of the second image feature selected by the second non-information weight.

[0130] Step S305 involves merging the first target feature and the second target feature to obtain merged features; and then performing spatial reconstruction on the merged features to obtain spatially reconstructed features. For details, please refer to [link to details]. Figure 1 Step S105 of the illustrated embodiment will not be described again here.

[0131] Step S306 involves training the classifier based on spatial reconstruction features; acquiring the target cell image, inputting the target cell image into the trained classifier, and obtaining the classification result corresponding to the target cell image. For details, please refer to [link to relevant documentation]. Figure 1 Step S106 of the illustrated embodiment will not be described again here.

[0132] The image classification method provided in this invention effectively filters and distinguishes important and unimportant information in an image by multiplying a second image feature by information weights and non-information weights. This operation helps the model focus on processing the most meaningful information in the current cell image, thereby improving the model's efficiency and performance. Distinguishing between the second information feature and the second non-information feature allows the model to more finely control the amount and quality of information processed. After generating the second information feature and the second non-information feature, the model can more effectively adapt to specific task requirements. By effectively processing these two types of information, the model's performance and generalization ability in various visual tasks can be significantly improved. Through the separation of information features and non-information features, the model can be more efficient in processing large-scale image data. This optimization helps reduce the model's computational burden and resource consumption, improving overall processing speed and real-time performance.

[0133] This embodiment provides an image classification method that can be used in computer devices, such as desktop computers and laptops. Figure 4 This is a flowchart of an image classification method according to an embodiment of the present invention, such as...Figure 4 As shown, the process includes the following steps:

[0134] Step S401: Obtain the original cell image; and segment the original cell image to obtain multiple image patches. For details, please refer to [link to relevant documentation]. Figure 1 Step S101 of the illustrated embodiment will not be described again here.

[0135] Step S402 involves extracting the corresponding first image features from the original cell image and extracting the corresponding second image features from multiple image patches. For details, please refer to [link to relevant documentation]. Figure 3 Step S302 of the illustrated embodiment will not be described again here.

[0136] Step S403: Perform weight separation processing on the first image feature and the second image feature to obtain the first information weight and the first non-information weight corresponding to the first image feature, and the second information weight and the second non-information weight corresponding to the second image feature; wherein, the information weight is used to characterize the spatial content portion of the image feature that is useful and expressive, and the non-information weight is used to characterize the spatial content portion of the image feature that is less or non-informative. For details, please refer to... Figure 2 Step S203 of the illustrated embodiment and Figure 3 Step S303 of the illustrated embodiment will not be described again here.

[0137] Step S404: Create a first target feature using the first image features, the first information weight, and the first non-information weight; and create a second target feature using the second image features, the second information weight, and the second non-information weight. For details, please refer to [link to details]. Figure 2 Step S204 of the illustrated embodiment and Figure 3 Step S304 of the illustrated embodiment will not be described again here.

[0138] Step S405: Merge the first target feature and the second target feature to obtain merged features; and perform spatial reconstruction on the merged features to obtain spatially reconstructed features.

[0139] Specifically, step S405 includes:

[0140] Step S4051: Use a convolutional layer to fuse the first information feature and the second information feature to obtain the first fused feature; and fuse the first non-information feature and the second non-information feature to obtain the second fused feature.

[0141] The first fused feature contains combined information from the first information feature and the second information feature. Specifically, a convolutional layer is used to fuse the first information feature and the second information feature to obtain the first fused feature. The convolutional layer can be viewed as a tool that combines two features, and its parameters are learnable weights.

[0142] The second fused feature incorporates the combined information of the first and second non-information features. Specifically, a convolutional layer is used to fuse the first and second non-information features to obtain the second fused feature.

[0143] Step S4052: The first fusion feature and the second fusion feature are merged, and the merged feature is convolved to obtain the merged feature.

[0144] The first and second fused features are merged and connected along the channel dimension to form a larger feature map. The merged feature map is then subjected to convolution processing. This convolution operation involves selecting appropriate kernel size, stride, padding, and activation function to effectively extract and integrate feature information. The final merged feature is the result of this convolution process; it integrates information from both the first and second fused features and represents a more abstract and higher-level feature representation.

[0145] Step S4053: Perform weight separation processing on the merged features to obtain the third information weight and the third non-information weight corresponding to the merged features.

[0146] The merged features are normalized to obtain the corresponding third standard features. These third standard features are then weighted to obtain the corresponding third standard weights. These third standard weights are mapped to a preset interval using a preset function to obtain the corresponding third mapping weights. Weights in the third mapping weights that are above a preset threshold are defined as third information weights, and weights in the third mapping weights that are below a preset threshold are defined as third non-information weights. In other words, third information weights are the portion of the third mapping weights above the preset threshold, containing useful and expressive spatial content. Third non-information weights are the portion of the third mapping weights below the preset threshold, containing little or no information and spatial redundancy.

[0147] Step S4054: Multiply the merged feature by the third information weight to obtain the third information feature, and multiply the merged feature by the third non-information weight to obtain the third non-information feature.

[0148] Multiply the merged features by the third information weights to obtain the third information features. This can be understood as extracting the informational value from the merged features, selected by the third information weight. Multiplying the merged features by the third non-information weight yields the third non-information feature. Similarly, it can be understood as extracting the non-information part selected by the third non-information weight from the merged features.

[0149] Step S4055: The third information feature is split into the first information sub-feature and the second information sub-feature, and the third non-information feature is split into the third information sub-feature and the fourth information sub-feature.

[0150] First information sub-features From the third information feature This is a subset separated from the others; this subset may contain information of a specific type or attribute, such as certain visual features of a particular object. Second information sub-features From the third information feature Another part separated from the first information sub-feature, this part of the feature can contain information of different types or attributes, and can be combined with the first information sub-feature. To some extent, they complement or supplement each other.

[0151] Third information sub-features From the third non-information feature The portion separated from the non-information part, although derived from the non-information part, may still contain some useful information or a portion with relatively small errors due to the splitting process. Fourth information sub-feature From the third non-information feature Another part separated from the data may contain another type of useless information, such as noise, background, or other content irrelevant to the task.

[0152] Step S4056: Add the first information sub-feature and the fourth information sub-feature element by element to obtain the first cross feature; and add the second information sub-feature and the third information sub-feature element by element to obtain the second cross feature.

[0153] The first cross feature combines information from different subsets of the third information feature and the third non-information feature by adding them element-wise. Specifically, the first cross feature is obtained by adding the first information sub-feature and the fourth information sub-feature element-wise.

[0154] The second cross feature also combines information from different subsets of the third information feature and the third non-information feature by adding them element-wise. Specifically, the corresponding elements in the second information sub-feature and the third information sub-feature are added together to obtain the second cross feature.

[0155] Step S4057: Merge the first intersection feature and the second intersection feature to obtain the spatial reconstruction feature.

[0156] The first and second intersection features are concatenated to form the final spatial reconstruction features. Specifically, this can be done based on... To express.

[0157] Step S406 involves training the classifier based on spatial reconstruction features; acquiring the target cell image, inputting the target cell image into the trained classifier, and obtaining the classification result corresponding to the target cell image. For details, please refer to [link to relevant documentation]. Figure 1 Step S106 of the illustrated embodiment will not be described again here.

[0158] The image classification method provided in this invention fuses first and second information features, as well as first and second non-information features, through convolutional layers, effectively integrating feature information from different sources and with different properties. This fusion helps to comprehensively utilize the advantages of multiple feature sources, improving the model's ability to express and analyze complex image information. Merging the first and second fused features and then processing them through convolution to obtain merged features allows the model to richly express image features from multiple perspectives. This information complementarity and richness helps reduce information loss and error propagation, improving the model's comprehensive understanding and processing capabilities of image content. Processing the stitched features through convolutional layers effectively extracts and enhances the spatial information of the features. Convolutional operations have local perception capabilities and parameter sharing characteristics, effectively reducing the number of parameters and computational complexity, while improving the accuracy and stability of feature representation. By performing weight separation processing on the merged features to generate third information weights and third non-information weights respectively, the influence of different types of features in subsequent processing can be finely controlled. This refined feature weighting helps the model more effectively distinguish and utilize important and unimportant information in images, thereby improving the model's performance and robustness in handling complex visual tasks. Multiplying the merged features by the third information weight and the third non-information weight respectively generates third information features and third non-information features. Further decomposing these into multiple information sub-features and non-information sub-features enhances the model's understanding of different levels and angles of image content, enabling the model to analyze and process input data more comprehensively. By element-wise summing the first and fourth information sub-features, and element-wise summing the second and third information sub-features, first and second cross features are generated. This cross feature generation captures the interactions and correlations between different information subsets, helping to improve the model's performance and generalization ability in complex scenes. Merging the first and second cross features generates spatial reconstruction features, integrating information from different information sub-features and cross features to form a more comprehensive and complete feature representation. This spatial reconstruction feature helps improve the model's understanding of the overall structure and spatial layout of the image, thereby improving the model's effectiveness and accuracy in image processing tasks.

[0159] This embodiment also provides an image classification device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0160] This embodiment provides an image classification device, such as... Figure 5 As shown, it includes:

[0161] The acquisition module 501 is used to acquire the original cell image and to segment the original cell image to obtain multiple image blocks.

[0162] The extraction module 502 is used to extract corresponding first image features from the original cell image and corresponding second image features from multiple image blocks.

[0163] The processing module 503 is used to perform weight separation processing on the first image feature and the second image feature to obtain the first information weight and the first non-information weight corresponding to the first image feature, and the second information weight and the second non-information weight corresponding to the second image feature; wherein, the information weight is used to characterize that the image feature contains a target space content part, and the non-information weight is used to characterize that the image feature does not contain a target space content part or the proportion of the target space content part is less than a preset value.

[0164] The creation module 504 is used to create a first target feature using a first image feature, a first information weight, and a first non-information weight; and to create a second target feature using a second image feature, a second information weight, and a second non-information weight.

[0165] The merging module 505 is used to merge the first target feature and the second target feature to obtain the merged feature; and to perform spatial reconstruction on the merged feature to obtain the spatially reconstructed feature.

[0166] The classification module 506 is used to train the classifier based on spatial reconstruction features; and to acquire the target cell image, input the target cell image into the trained classifier, and obtain the classification result corresponding to the target cell image.

[0167] In some alternative implementations, the processing module 503 includes:

[0168] The first normalization submodule is used to normalize the first image features to obtain the corresponding first standard features.

[0169] The first weighting submodule is used to weight the first standard features to obtain the corresponding first standard weights.

[0170] The first mapping submodule is used to map the first standard weights to a preset interval through a preset function to obtain the corresponding first mapping weights.

[0171] The first determining submodule is used to obtain a preset threshold; and to determine the weights in the first mapping weights that are higher than the preset threshold as first information weights, and to determine the weights in the first mapping weights that are lower than the preset threshold as first non-information weights.

[0172] In some optional implementations, the first target feature includes a first informational feature and a first non-informational feature; the creation module 504 includes:

[0173] The first calculation submodule is used to multiply the first image feature by the first information weight to obtain the first information feature, and to multiply the first image feature by the first non-information weight to obtain the first non-information feature.

[0174] In some alternative implementations, the extraction module 502 includes:

[0175] The acquisition submodule is used to acquire the position encoding vectors corresponding to multiple image blocks, and to perform linear projection on the multiple image blocks to obtain the projection vectors corresponding to the multiple image blocks.

[0176] The computation submodule is used to add the projection vector and the position encoding vector to obtain the second image features corresponding to multiple image blocks.

[0177] In some optional implementations, the processing module 503 further includes:

[0178] The first processing submodule is used to process the second image features according to the multi-head self-attention mechanism to obtain the multi-head feature vector; wherein the multi-head feature vector is used to represent the information of the second image features at different points of interest.

[0179] The second normalization submodule is used to normalize the multi-head feature vector to obtain the corresponding second standard feature.

[0180] The second weighting submodule is used to weight the second standard features to obtain the corresponding second standard weights.

[0181] The second mapping submodule is used to map the second standard weights to a preset interval through a preset function to obtain the corresponding second mapping weights.

[0182] The second determining submodule is used to determine the weights in the second mapping weights that are higher than a preset threshold as second information weights, and to determine the weights in the second mapping weights that are lower than the preset threshold as second non-information weights.

[0183] In some alternative implementations, the processing submodule includes:

[0184] The acquisition unit is used to acquire the pre-defined query weight matrix, key weight matrix, and value weight matrix.

[0185] The first calculation unit is used to multiply the second image features by the query weight matrix, the key weight matrix, and the value weight matrix respectively to obtain the corresponding query vector, key vector, and value vector.

[0186] The second computational unit is used to calculate the dot product of the query vector and the key vector, and to determine the dot product as the attention score.

[0187] The third computational unit is used to perform weighted summation of attention scores to obtain the single-head feature vector corresponding to each attention head.

[0188] The merging unit is used to merge single-head feature vectors and perform a linear transformation on the merged feature vectors to obtain multi-head feature vectors corresponding to multiple attention heads.

[0189] In some optional implementations, the second target feature includes a second informational feature and a second non-informational feature; the creation module 504 further includes:

[0190] The second calculation submodule is used to multiply the second image features by the second information weights to obtain the second information features, and to multiply the second image features by the second non-information weights to obtain the second non-information features.

[0191] In some alternative implementations, the merging module 505 includes:

[0192] The fusion submodule is used to fuse the first information feature and the second information feature using a convolutional layer to obtain the first fused feature; and to fuse the first non-information feature and the second non-information feature to obtain the second fused feature.

[0193] The first merging submodule is used to merge the first fused feature and the second fused feature, and to perform convolution processing on the merged feature to obtain the merged feature.

[0194] In some alternative implementations, the merging module 505 further includes:

[0195] The second processing submodule is used to perform weight separation processing on the merged features to obtain the third information weight and the third non-information weight corresponding to the merged features.

[0196] The third calculation submodule is used to multiply the merged feature by the third information weight to obtain the third information feature, and to multiply the merged feature by the third non-information weight to obtain the third non-information feature.

[0197] The splitting submodule is used to split the third information feature into the first information sub-feature and the second information sub-feature, and to split the third non-information feature into the third information sub-feature and the fourth information sub-feature.

[0198] The fourth calculation submodule is used to add the first information sub-feature and the fourth information sub-feature element by element to obtain the first cross feature; and to add the second information sub-feature and the third information sub-feature element by element to obtain the second cross feature.

[0199] The second merging submodule is used to merge the first cross feature and the second cross feature to obtain the spatial reconstruction feature.

[0200] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0201] In this embodiment, the image classification device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0202] The image classification device provided in this invention can extract multiple image patches by acquiring and segmenting the original cell image, which helps in processing large-scale data and improving processing efficiency. Extracting image features from the original cell image and its segmented image patches captures visual information at different scales and levels, which is beneficial for subsequent analysis and processing. Weighting the image features separately, processing information weights and non-information weights, helps reduce the influence of noise and redundant information, improving the expressive power of features and the accuracy of classification. Creating target features using information weights and non-information weights, and merging target features from different sources, can comprehensively utilize information at different levels and types, improving the classification and recognition effect. Training the classifier using spatial reconstruction features can fully utilize the spatial information of the image, improving the classifier's generalization ability and accuracy for target cell images. Applying the trained classifier to target cell images enables rapid classification and recognition of unknown cell image data, effectively improving the accuracy and efficiency of cell image classification and recognition tasks.

[0203] This invention also provides a computer device having the above-described features. Figure 5 The image classification device shown.

[0204] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 6As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 6 Take a processor 10 as an example.

[0205] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0206] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0207] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0208] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0209] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0210] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0211] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0212] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0213] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. An image classification method, characterized in that, The method includes: Acquire the original cell image; and segment the original cell image to obtain multiple image blocks; Extract the corresponding first image feature from the original cell image, and extract the corresponding second image feature from the plurality of image blocks; The first image feature and the second image feature are subjected to weight separation processing to obtain a first information weight and a first non-information weight corresponding to the first image feature, and a second information weight and a second non-information weight corresponding to the second image feature; wherein, the information weight is used to characterize that the image feature contains a target space content part, and the non-information weight is used to characterize that the image feature does not contain the target space content part or the proportion of the target space content part is less than a preset value. A first target feature is created using the first image feature, the first information weight, and the first non-information weight; and a second target feature is created using the second image feature, the second information weight, and the second non-information weight. The first target feature and the second target feature are merged to obtain a merged feature; and the merged feature is spatially reconstructed to obtain a spatially reconstructed feature. The classifier is trained based on the spatial reconstruction features; and the target cell image is acquired and input into the trained classifier to obtain the classification result corresponding to the target cell image.

2. The method according to claim 1, characterized in that, The first image feature is subjected to weight separation processing to obtain the first information weight and the first non-information weight corresponding to the first image feature, including: The first image features are normalized to obtain the corresponding first standard features; The first standard features are weighted to obtain the corresponding first standard weights; The first standard weight is mapped to a preset interval through a preset function to obtain the corresponding first mapped weight; Obtain a preset threshold; and determine the weights in the first mapping weights that are higher than the preset threshold as the first information weights, and determine the weights in the first mapping weights that are lower than the preset threshold as the first non-information weights.

3. The method according to claim 2, characterized in that, The first target feature includes a first information feature and a first non-information feature; Creating a first target feature using the first image feature, the first information weight, and the first non-information weight includes: The first information feature is obtained by multiplying the first image feature by the first information weight, and the first non-information feature is obtained by multiplying the first image feature by the first non-information weight.

4. The method according to claim 1, characterized in that, Extracting corresponding second image features from the plurality of image blocks includes: Obtain the position encoding vectors corresponding to the plurality of image blocks, and perform linear projection on the plurality of image blocks to obtain the projection vectors corresponding to the plurality of image blocks; The projection vector and the position encoding vector are added together to obtain the second image feature corresponding to the plurality of image blocks.

5. The method according to claim 4, characterized in that, The second image feature is subjected to weight separation processing to obtain the second information weight and the second non-information weight corresponding to the second image feature, including: The second image feature is processed according to the multi-head self-attention mechanism to obtain a multi-head feature vector; wherein, the multi-head feature vector is used to characterize the information of the second image feature at different points of interest; The multi-head feature vector is normalized to obtain the corresponding second standard feature; The second standard features are weighted to obtain the corresponding second standard weights; The second standard weight is mapped to a preset interval through a preset function to obtain the corresponding second mapped weight; The weights in the second mapping weights that are higher than a preset threshold are determined as the second information weights, and the weights in the second mapping weights that are lower than the preset threshold are determined as the second non-information weights.

6. The method according to claim 5, characterized in that, The step of processing the second image features according to the multi-head self-attention mechanism to obtain the multi-head feature vector includes: Obtain the pre-defined query weight matrix, key weight matrix, and value weight matrix; The second image feature is multiplied by the query weight matrix, the key weight matrix, and the value weight matrix respectively to obtain the corresponding query vector, key vector, and value vector; Calculate the dot product of the query vector and the key vector, and determine the dot product as the attention score; The attention scores are weighted and summed to obtain the single-head feature vectors corresponding to each attention head; The single-head feature vectors are merged, and the merged feature vectors are linearly transformed to obtain the multi-head feature vectors corresponding to multiple attention heads.

7. The method according to claim 6, characterized in that, The second target features include second information features and second non-information features; Creating a second target feature using the second image features, the second information weights, and the second non-information weights includes: The second information feature is obtained by multiplying the second image feature by the second information weight, and the second non-information feature is obtained by multiplying the second image feature by the second non-information weight.

8. The method according to claim 1, characterized in that, The first target feature includes a first information feature and a first non-information feature; the second target feature includes a second information feature and a second non-information feature. The merging of the first target feature and the second target feature to obtain the merged feature includes: The first information feature and the second information feature are fused using a convolutional layer to obtain a first fused feature; and the first non-information feature and the second non-information feature are fused to obtain a second fused feature. The first fusion feature and the second fusion feature are merged, and the merged feature is then convolved to obtain the merged feature.

9. The method according to claim 8, characterized in that, Spatial reconstruction is performed on the merged features to obtain spatially reconstructed features, including: The merged features are subjected to weight separation processing to obtain the third information weight and the third non-information weight corresponding to the merged features; The third information feature is obtained by multiplying the merged feature by the third information weight, and the third non-information feature is obtained by multiplying the merged feature by the third non-information weight. The third information feature is divided into a first information sub-feature and a second information sub-feature, and the third non-information feature is divided into a third information sub-feature and a fourth information sub-feature; The first information sub-feature and the fourth information sub-feature are added element by element to obtain the first cross feature; and the second information sub-feature and the third information sub-feature are added element by element to obtain the second cross feature. The first intersection feature and the second intersection feature are merged to obtain the spatial reconstruction feature.

10. An image classification device, characterized in that, The device includes: An acquisition module is used to acquire a raw cell image and to segment the raw cell image to obtain multiple image blocks. An extraction module is used to extract corresponding first image features from the original cell image and corresponding second image features from the plurality of image blocks; The processing module is used to perform weight separation processing on the first image feature and the second image feature to obtain a first information weight and a first non-information weight corresponding to the first image feature, and a second information weight and a second non-information weight corresponding to the second image feature; wherein, the information weight is used to characterize that the image feature contains a target space content part, and the non-information weight is used to characterize that the image feature does not contain the target space content part or the proportion of the target space content part is less than a preset value. A creation module is configured to create a first target feature using the first image feature, the first information weight, and the first non-information weight; and to create a second target feature using the second image feature, the second information weight, and the second non-information weight. The merging module is used to merge the first target feature and the second target feature to obtain a merged feature; and to perform spatial reconstruction on the merged feature to obtain a spatially reconstructed feature; The classification module is used to train a classifier based on the spatial reconstruction features; and to acquire a target cell image, input the target cell image into the trained classifier, and obtain the classification result corresponding to the target cell image.

11. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the image classification method according to any one of claims 1 to 9 by executing the computer instructions.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the image classification method according to any one of claims 1 to 9.

13. A computer program product, characterized in that, Includes computer instructions for causing a computer to perform the image classification method according to any one of claims 1 to 9.

Images