Visual recognition method, device, equipment, storage medium and program product

Abstract

A visual recognition method, device, equipment, storage medium and program product are disclosed, pre-processing is performed on to-be-recognized visual data to obtain input data, the input data is encoded by an image embedding module of a visual recognition model to obtain image serialized feature representation, an index matrix is obtained through a cross-layer attention extraction mechanism of multiple encoder layers of a visual transformer; based on the index matrix, a cross-layer feature subset is screened from the image serialized feature representation, the feature subset is input into a feature enhancement module, feature weighting fusion is performed on the feature subset by the feature enhancement module to obtain fusion features containing global semantics and fine-grained features; and classification recognition of the to-be-recognized visual data is performed based on the fusion features. By adopting the embodiment of the present application, the efficiency and feature capturing capability of fine-grained recognition can be effectively balanced, the computing overhead is reduced, the recognition accuracy and robustness are improved, and the multi-scene visual recognition demand is adapted.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more particularly to a visual recognition method, apparatus, device, storage medium, and program product. Background Technology

[0002] Visual recognition refers to intelligent technology that uses computer technology to detect, classify, and distinguish targets in images or videos. It is one of the application directions of artificial intelligence and is widely used in scenarios such as industrial quality inspection, smart retail, and security monitoring. Image classification is the foundation of visual recognition, and its difficulty lies in fine-grained visual classification (FGVC). It not only needs to accurately identify the major categories of targets, but also needs to distinguish the subtle differences between subcategories (such as the differences in beak shape and feather texture between different species of birds).

[0003] Traditional Convolutional Neural Networks (CNNs) are limited by their local receptive fields, making it difficult to establish cross-regional semantic relationships. Vision Transformers (ViTs) effectively compensate for this deficiency by capturing global relationships through self-attention mechanisms. However, in the FGVC task, their training efficiency is low due to the quadratic complexity of image resolution, and they do not pay enough attention to local fine-grained features, requiring additional modules for compensation. In addition, while existing state-of-the-art fine-grained recognition models (such as TransFG and SIM Trans) have improved recognition accuracy by introducing complex modules, they have significantly increased training time and computational costs, failing to balance efficiency and accuracy. Furthermore, some models only extract single-layer features or lack effective feature transformation and activation mechanisms, resulting in limited feature representation capabilities and insufficient robustness to image changes, making it difficult to meet the demands of efficient and high-precision recognition in practical applications. Summary of the Invention

[0004] The purpose of this invention is to provide a visual recognition method, apparatus, device, storage medium, and program product that can effectively balance the efficiency of fine-grained recognition with feature capture capability, reduce computational overhead, improve recognition accuracy and robustness, and adapt to visual recognition needs in multiple scenarios.

[0005] To achieve the above objectives, embodiments of the present invention provide a visual recognition method, comprising: Acquire the visual data to be identified, and preprocess the visual data to be identified to obtain preprocessed input data; The input data is encoded by the image embedding module of the visual recognition model to obtain an image serialization feature representation; The image serialization feature representation is input into the visual transformer of the visual recognition model, and an index matrix is ​​obtained through the cross-layer attention extraction mechanism of multiple encoder layers of the visual transformer; Based on the index matrix, a cross-level feature subset is selected from the image serialization feature representation; The feature subset is input into the feature enhancement module of the visual recognition model, and the feature enhancement module performs feature weighted fusion on the feature subset to obtain a fused feature that includes global semantics and fine-grained features; The visual data to be identified is classified and recognized based on the fusion features.

[0006] As an improvement to the above scheme, the step of encoding the input data through the image embedding module of the visual recognition model to obtain an image serialization feature representation includes: The input data is processed into image blocks to obtain multiple image block features; A classification label is concatenated before the sequence composed of multiple image patch features, and positional encoding features are added to the classification label and each image patch feature to obtain an image serialization feature representation.

[0007] As an improvement to the above scheme, the index matrix obtained through the cross-layer attention extraction mechanism of multiple encoder layers of the visual transformer includes: The image serialization feature representation is sequentially input into multiple encoder layers of the visual transformer to extract the attention weights of the classification labels for all image patches using the encoder layers; The location index of the image patch most concerned by the classification label in each encoder layer is determined based on the attention weights. The image block position indices obtained from each encoder layer are offset-corrected, and all the corrected image block position indices are integrated to obtain an index matrix.

[0008] As an improvement to the above scheme, the feature enhancement module performs feature weighting and fusion on the feature subset to obtain a fused feature containing global semantics and fine-grained features, including: The feature subset is input to the feature enhancement module, which performs nonlinear mapping and normalization on the feature subset to obtain the weight coefficient of each feature in the feature subset. The feature subset is weighted and summed based on the weight coefficients to obtain enhanced fine-grained features. The fine-grained features are fused with the image serialization feature representation through residual connections to obtain fused features that include global semantic features and fine-grained features.

[0009] As an improvement to the above scheme, the training process of the visual recognition model includes: Configure training parameters and initialize the network parameters of the visual recognition model; Obtain a visual recognition dataset and divide the visual recognition dataset into training set data and test set data; The training set data is input into the initialized visual recognition model, and the fusion features output by the visual recognition model are obtained, and the sample prediction category is obtained based on the fusion features; The loss value is calculated based on the predicted category and the true category of the sample. The network parameters of the visual recognition model are updated based on the loss value until the preset number of training steps is completed or the model verification accuracy converges, thus obtaining the trained visual recognition model.

[0010] As an improvement to the above scheme, the training parameters include training execution parameters and optimization strategy parameters; wherein, the training execution parameters include the total number of training steps and model validation parameters, and the optimization strategy parameters include the learning rate and the number of warm-up mechanism steps.

[0011] As an improvement to the above scheme, the learning rate is dynamically adjusted through a cosine annealing scheduling strategy; wherein the cosine annealing scheduling strategy indicates the variation pattern of the learning rate in different preheating stages.

[0012] To achieve the above objectives, embodiments of the present invention also provide a visual recognition device, comprising: The data preprocessing module is used to acquire visual data to be recognized and to preprocess the visual data to be recognized to obtain preprocessed input data. The encoding module is used to encode the input data through the image embedding module of the visual recognition model to obtain an image serialization feature representation; The location index generation module is used to input the image serialization feature representation into the visual transformer of the visual recognition model, and obtain the index matrix through the cross-layer attention extraction mechanism of multiple encoder layers of the visual transformer; The feature subset generation module is used to filter cross-level feature subsets from the image serialization feature representation based on the index matrix; The feature generation module is used to input the feature subset into the feature enhancement module of the visual recognition model, and perform feature weighted fusion on the feature subset through the feature enhancement module to obtain fused features containing global semantics and fine-grained features; The classification and recognition module is used to classify and recognize the visual data to be recognized based on the fused features.

[0013] To achieve the above objectives, embodiments of the present invention also provide a visual recognition device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the visual recognition method as described in any of the above embodiments.

[0014] To achieve the above objectives, embodiments of the present invention also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the visual recognition method as described in any of the above embodiments.

[0015] To achieve the above objectives, embodiments of the present invention also provide a computer program product, including computer instructions, which, when executed by a processor, implement the visual recognition method as described in any of the above embodiments.

[0016] Compared to existing technologies, the visual recognition method, apparatus, device, storage medium, and program products disclosed in this invention standardize the input data format through preprocessing, implement serialized feature encoding of visual data through an image embedding module, and then accurately locate key regions of cross-layer attention using a cross-layer attention extraction mechanism of a multi-encoder layer in a visual transformer. This allows for the selection of highly discriminative cross-layer feature subsets, followed by weighted fusion operations through a feature enhancement module to achieve an organic combination of fine-grained features and global semantic features. Finally, classification and recognition are completed based on the fused features. This invention not only accurately captures fine-grained visual features through cross-layer attention extraction, solving the problem of insufficient attention to local key features in traditional recognition methods, but also ensures the integrity of global semantic information through feature weighted fusion, effectively improving the accuracy and robustness of visual recognition. Simultaneously, while maintaining recognition performance, it simplifies the calculation process of feature selection and enhancement, improving the execution efficiency of the recognition method, and is applicable to various fine-grained visual recognition scenarios. Attached Figure Description

[0017] Figure 1 This is a structural block diagram of the visual recognition model provided in the embodiments of the present invention; Figure 2 This is a structural block diagram of the feature enhancement module provided in an embodiment of the present invention; Figure 3 This is a flowchart of a visual recognition method provided in an embodiment of the present invention; Figure 4 This is a comparative schematic diagram of the original image and the non-overlapping cut-out image provided in the embodiments of the present invention; Figure 5 This is a comparative schematic diagram of non-overlapping and overlapping cut-out images provided in an embodiment of the present invention; Figure 6This is the cosine annealing learning rate curve provided in the embodiments of the present invention; Figure 7 This is a schematic diagram of a dataset subclass sample provided in an embodiment of the present invention; Figure 8 This is a comparison chart of the model accuracy of the three models provided in this embodiment of the invention after 10,000 training steps; Figure 9 This is a comparison chart of the model loss of the three models provided in this embodiment of the invention after 2500 training steps; Figure 10 These are attention visualizations of the three models provided in this embodiment of the invention; Figure 11 This is a structural block diagram of a visual recognition device provided in an embodiment of the present invention; Figure 12 This is a structural block diagram of a visual recognition device provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The visual recognition model described in this invention is a hierarchical part-attention and feature reinforcement vision transformer (PAFR-ViT). Unlike existing fine-grained image classification methods (such as TransFG) that independently perform part selection within a single Transformer Encoder layer, this invention introduces a hierarchical part selection module across multiple Transformer Encoder layers of the ViT (Vision Transformer). This module selects discriminative local image regions layer by layer and establishes cross-layer local attention relationships. Furthermore, by setting a part attention module, this invention jointly models and aggregates local attention information from different Transformer Encoder layers, thereby achieving synergistic enhancement of cross-layer local semantic information, rather than relying solely on the local feature representation of a single layer. Based on this, a Feature Reinforcement Module (FRM) is introduced. The Part Attention module filters a subset of salient region features through cross-layer indexing. The FRM performs adaptive weighted fusion on this subset to complete the feature selection and fusion. Through the above structural design, this invention effectively overcomes the problems of unstable local feature semantics and layer-by-layer weakening of deep features in the prior art, and improves the performance and robustness of fine-grained image classification.

[0020] See Figure 1 , Figure 1 This is a block diagram of the visual recognition model structure provided in an embodiment of the present invention. The visual recognition model includes an image input layer, an image embedding module, a visual transformer (including a multi-layer Transformer Encoder and supporting sub-modules), a feature enhancement module (FRM), a classification head, and a loss calculation module. The functions of each part are as follows: 1) Image input layer (Image): Receives the raw visual data to be recognized as the initial input for model processing, providing the raw data foundation for subsequent preprocessing and feature encoding.

[0021] 2) The image embedding module (Linear Projection of Flattened Patches) converts the two-dimensional preprocessed image into a one-dimensional serialized feature representation that can be processed by the visual transformer, achieving image-vector format adaptation while preserving the local features and spatial location information of the image. It should be noted that the numbering "0-9" between the image embedding module and the Transformer encoder represents a visual representation of the image serialized feature sequence output by the image embedding module. Position 0 represents the CLS Token (Classification Token), a learnable vector used to aggregate global semantic information. Positions 1-9 represent the image patch features (Patch Tokens) obtained after image segmentation and linear projection, with each number corresponding to the position index of an image patch.

[0022] 3) The visual transformer receives the image sequenced feature representation output by the image embedding module. Through multi-layer encoding and cross-layer attention operations, it captures the semantic relationships between features and mines consistent local discriminative information across layers, providing multi-level feature support and accurate local region location information for subsequent feature selection and enhancement. It includes a multi-layer Transformer encoder, a Part Selection Module (MMT) and a Part Attention Module (MAI) corresponding to each encoder layer, specifically: ① The Transformer Encoder captures the semantic relationships between vectors in the image serialization features through a self-attention mechanism, extracting higher-level feature representations layer by layer, providing a multi-semantic feature foundation for subsequent cross-layer attention extraction; for example, each encoder layer includes multi-head attention, feedforward network (FFN), residual connection and layer normalization, following the classic ViT architecture to ensure deep semantic modeling of features, and each encoder layer is equipped with a Part Selection Module, which extracts the attention weights of the layer's CLS Token for all image patches after completing the self-attention calculation, providing single-layer local attention information for cross-layer attention extraction; ② The Part Selection Module is used to accurately locate the local discrimination region that the CLS Token is most interested in in each Transformer Encoder layer, and output the image patch position index of that layer to provide the coordinates of the key region in a single layer for cross-layer attention integration; ③ Part Attention is used to jointly model and aggregate the local location indices output by the multi-level Part Selection Module to generate an index matrix, while filtering out a subset of salient region features across levels to achieve synergistic enhancement of multi-level local attention information.

[0023] It should be noted that the "×(L" in the figure 1) indicates that the module structure will be repeatedly stacked. The first layer is a typical layer stacking design in the Visual Transformer (ViT) model. This repeated stacking design is to continuously capture and retain the significant local features of interest in each layer from the encoder outputs of multiple different semantic levels. It provides multi-dimensional input for subsequent cross-level feature filtering and enhancement, allowing the model to integrate detailed information from different levels with global semantics.

[0024] 4) Feature Enhancement Module (FRM) is used to perform targeted enhancement and residual fusion on the subset of salient region features selected by Part Attention, generating fused features containing global semantics and fine-grained features, thereby improving the discriminativeness of features and the robustness of the model.

[0025] 5) The Classification Head maps the fused features output by the Feature Enhancement Module (FRM) to the target class dimension, outputting the class prediction result of the image to be recognized, completing the final feature-to-class transformation. It can consist of two fully connected (Linear) layers: the first layer maps the fused features to the intermediate dimension, and the second layer maps to the target class number. It should be noted that the pre-alignment feature is usually composed of multiple normalization / mapping layers (illustrated as stacked blocks in the figure), which preprocess the input features of the Classification Head.

[0026] 6) Loss calculation module: The output layer does not have an activation function. The results are directly passed to the loss calculation module. For example, the difference between the predicted class and the true class is calculated by cross-entropy loss, which is used for model parameter updates. Cross-entropy loss is used as the main loss function to measure the difference between the predicted probability distribution of the class and the true label, guiding the model to optimize classification accuracy. Contrastive loss can also be introduced to help improve the discriminativeness of fine-grained features and further optimize classification performance.

[0027] While ViT-based models possess long-range modeling capabilities, they often struggle to extract discriminative information from regions when handling fine-grained classification or recognition tasks in complex scenes. Therefore, this invention proposes a Feature Enhancement Module (FRM) to mine and enhance local feature information related to key semantic regions, thereby improving the discriminative power of the overall representation and the robustness of the model. To uncover category-discriminating visual cues within local regions, such as key parts of objects or specific structures in the background, while individual visual cues may not provide complete semantic information, they play a crucial role in the overall representation. To achieve this, this invention, based on the original feature sequence, references features corresponding to the maximum attention region at each layer obtained through the attention mechanism. These features are then modeled and fused in a learnable manner to achieve information enhancement.

[0028] See Figure 2 , Figure 2 This is a structural block diagram of the Feature Enhancement Module (FRM) provided in this embodiment of the invention. This module enhances fine-grained features and preserves global semantics by performing targeted weighting and fusion on a subset of salient region features obtained from cross-layer attention filtering. Its internal process and the functions of each sub-module are as follows: 1) Input: This is used to receive a subset of cross-level salient region features from the output of the Part Attention module, serving as the initial data for module processing.

[0029] 2) The first linear layer is used to perform the first dimension mapping on the input features, adjusting the feature dimensions to fit the dimensions of subsequent nonlinear transformations; 3) Tangent activation function (tanh) is used to introduce nonlinear transformations to enhance the expressive power and discriminative power of features; 4) The first dropout layer is used to randomly drop some neurons during the training phase to prevent the model from overfitting and improve generalization ability; 5) The second linear layer is used to perform a second dimensionality mapping on the features output by the first dropout layer, converting the features into a single-dimensional weight vector; 6) The normalization function (Softmax) is used to normalize the weight vector to ensure that the sum of the weight coefficients of each feature is 1, so as to depict the contribution of different salient regions to the classification task. 7) The second dropout layer is used to further suppress overfitting and stabilize the learning process of the weight coefficients; 8) Feature weighted summation (Einsum) is used to perform weighted summation on a subset of salient features of the original input based on the weight coefficients of the output, to obtain an enhanced fine-grained feature vector, highlighting the role of highly discriminative features; 9) Layer Normalization is used to normalize the weighted and fused features, stabilize the feature distribution, avoid gradient vanishing or exploding, and improve the stability of model training.

[0030] 10) Output: The final output is a result that combines global semantics with enhanced fine-grained features. These features will be fused with the original encoder features through residual connections to provide more discriminative input for subsequent classification heads.

[0031] To ensure the efficiency of the visual recognition model PAFR-ViT and the reproducibility of experimental results, this invention established a corresponding computing environment on the Linux operating system. This environment mainly includes GPU (Graphics Processing Unit) acceleration hardware and integrates mainstream deep learning frameworks (such as PyTorch) and the Apex (Automatic Mixed Precision for PyTorch) distributed training tool. The experimental environment configuration data can be found in Table 1. It should be noted that the data in Table 1 is only an example; in practical applications, the configuration information can be adjusted as needed.

[0032] Table 1 Example of experimental environment configuration data

[0033] For example, this invention is configured with a GPU, specifically a Tesla V100-PCIE-16GB model. This is a professional-grade GPU designed specifically for high-performance computing. Its 16GB of video memory can fully support the multi-layer Transformer encoding and cross-layer attention calculations of the PAFR-ViT model, improving training and inference efficiency. The accompanying CPU (Central Processing Unit) is an Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz. This server-grade processor features multi-core, high-thread characteristics, enabling efficient completion of CPU-intensive tasks such as data preprocessing and model scheduling. It also includes 376GB of memory, which can cache large datasets, avoiding frequent disk I / O and ensuring smooth data loading and training processes.

[0034] This invention embodiment was first tested on a high-performance computing platform equipped with eight NVIDIA Tesla V100 graphics cards with 16GB of dedicated video memory, sufficient to meet the demands of high-parallel computing and large-scale data processing during deep learning training. The central processing unit is an Intel(R) Xeon(R) Gold 5118 (2.30GHz), coupled with 376GB of RAM, providing ample computing power and buffering resources for data preprocessing, multi-threaded scheduling, and large-scale tensor operations. The operating system used is Ubuntu 22.04.2 LTS, a long-term support version with strong stability, suitable for research applications. The system kernel version is 5.15.0-130-generic, optimized for NVIDIA drivers, specifically NVIDIA Driver 560.35.03, enabling efficient DMA data transfer between CUDA computing units and host memory. Furthermore, the system environment includes a basic development toolchain such as GCC installed via the APT package manager, providing support for compiling underlying libraries and building dependency environments.

[0035] To improve environment maintainability and avoid dependency conflicts, this invention uses Miniconda 25.1.1 to create an isolated Python virtual environment. Compared to the full version of Anaconda, it reduces disk usage by approximately 85% while retaining dependency resolution functionality, making it suitable for deployment on research servers with multiple projects running concurrently. The virtual environment is configured with the Python 3.8.0 interpreter, which strikes a balance in terms of compatibility with third-party libraries. The deep learning framework chosen is PyTorch 2.4.1, which, combined with CUDA Toolkit 11.8, fully utilizes the computing power of the V100 GPU, improving training speed and resource utilization. This virtual environment provides a reliable foundation for the model building, training, and evaluation processes in the following experiments.

[0036] Furthermore, to fully exploit the computational potential of GPUs and improve parallel efficiency during large-scale model training, this study constructed an Apex environment supporting mixed-precision and distributed training within the PyTorch framework. Apex is a performance optimization library officially maintained by NVIDIA. While ensuring numerical accuracy, it reduces memory usage and accelerates training speed through FP16 precision training, making it suitable for multi-GPU synchronous deep learning tasks. Table 2 lists the key dependency configurations required to build the Apex distributed environment.

[0037] Table 2. Key dependency configurations required to build the Apex distributed environment

[0038] For example, the Apex source code is obtained from NVIDIA's official open-source project on GitHub. During the build process, ensure that GCC / G++ 9.5.0 is correctly configured in the system environment to ensure compatibility with the C++ components in the Apex source code. In a Python 3.8.0 virtual environment, install Apex by compiling from source code, ensuring that its version 24.4.1 is fully aligned with the currently used CUDA 11.8 and PyTorch 2.4.1 to avoid compatibility issues during compilation. After completing the environment dependency installation and version alignment, Apex's core functions can be called via Python scripts. The mixed-precision training function is provided by the amp module, while the distributed training function is implemented through the encapsulated Distributed Data Parallel class.

[0039] See Figure 3 , Figure 3 This is a flowchart of a visual recognition method provided in an embodiment of the present invention, which includes steps S1 to S6.

[0040] S1. Obtain the visual data to be recognized and preprocess the visual data to be recognized to obtain preprocessed input data.

[0041] For example, the visual data to be identified refers to the original visual image data that needs to be classified and identified in a fine-grained manner using the visual recognition model of this invention. It serves as the basic data source for the model's feature extraction, encoding, and classification. For instance, it can be image data with subtle category differences, such as birds, flowers, vehicles, and aircraft engine blades, in a fine-grained image classification scenario. Preprocessing aims to unify the format of the original visual data, eliminate data noise, and improve the data-model compatibility, making the subsequent feature extraction and encoding processes of the model more efficient and stable. Preprocessing includes operations such as image size normalization, cropping, and pixel value standardization. For example, the original visual data can first be scaled using bilinear interpolation to uniformly scale original images of different sizes to a preset size. Then, the scaled images are cropped according to preset cropping rules (such as center cropping or random cropping) to obtain images of fixed sizes. Finally, the pixel values ​​of the cropped images are standardized based on preset mean and standard deviation to eliminate differences in pixel values ​​caused by factors such as lighting and equipment. This results in preprocessed input data with a unified format and regular features, laying a standardized data foundation for the subsequent encoding operations of the image embedding module.

[0042] S2. The input data is encoded by the image embedding module of the visual recognition model to obtain an image serialization feature representation.

[0043] For example, the image embedding module is the embedding module mentioned above used to convert a two-dimensional image into a one-dimensional token sequence representation. This module includes Patch Embedding, Position Embedding, and a learnable classification token (CLS token). It also supports overlapping or non-overlapping slicing strategies, enabling the transformation from the original image to the token representation and adapting to the input requirements of the Transformer structure.

[0044] Furthermore, step S2 specifically includes steps S21 to S22.

[0045] S21. Perform image block processing on the input data to obtain multiple image block features.

[0046] For example, image patch processing refers to dividing a two-dimensional input image into multiple local image patches according to a set patch size (e.g., 16×16) and stride through a convolution operation, and mapping each image patch to a fixed-dimensional vector feature. Patch Embedding can divide the image according to a set patch size and map the local regions of the image to a vector space through a convolution operation. This process is essentially equivalent to a convolution operation with a specified stride. See also... Figure 4 , Figure 4 This is a comparative diagram of the original image and the non-overlapping sliced ​​image provided in an embodiment of the present invention. Non-overlapping slices refer to dividing the image into regular grid blocks of uniform size that do not overlap, with no pixel overlap between the slices. In this way, the boundaries of each image block are clear, which is suitable for fast and uniform image segmentation, but it may disrupt the continuity of local features. For example, in fine-grained recognition, key discriminative information such as bird beaks may be abruptly divided into different blocks. Figure 4 The image on the left is the original fine-grained input image (taking a bird image as an example), which contains key discriminative information such as beak and feather texture. The image on the right is the image after non-overlapping slicing. The image is divided into regular grid blocks of uniform size that do not overlap. Each block corresponds to an image patch token. This slicing process is essentially equivalent to a convolution operation. When the non-overlapping strategy is used, the convolution stride is the same as the convolution kernel size, thereby achieving uniform slicing without overlap. This method can convert a two-dimensional image into a one-dimensional image patch sequence, adapting to the input format of the Transformer model, while preserving the local spatial features of the image.

[0047] See Figure 5 , Figure 5This is a comparative diagram of non-overlapping and overlapping image segments provided in this embodiment of the invention. Overlapping segments refer to partial pixel overlap between image segments. This is achieved by setting the convolution stride to be smaller than the kernel size. Overlapping regions enhance feature continuity and spatial coverage, allowing the model to better capture cross-segment discriminative information, such as the connection details between a bird's beak and head, thereby improving the accuracy of fine-grained image classification. When using an overlapping strategy, overlapping regions between image segments are obtained by manually setting the stride to be smaller than the kernel size, thus improving feature continuity and spatial coverage. This helps the model better mine regional discriminative information between image segments.

[0048] The image embedding process is shown in formula (1). The input channel of Conv2d is the hidden layer dimension D of Trans Former. The input image size is (B, C, H, W), and the output is a patch sequence of (B, N, D). Where B is the batch size, C is the number of channels, H and W are the height and width of the image, respectively, and N represents the number of patches.

[0049] (1); in, This represents the output features obtained after image patch embedding, i.e., image patch features, with dimensions (B, N, D). This represents a two-dimensional convolution operation that maps the input image x into a sequence of image patch features.

[0050] S22. A classification label is concatenated before the sequence composed of the features of the multiple image blocks, and positional encoding features are added to the classification label and each image block feature to obtain an image serialization feature representation.

[0051] For example, the classification token (CLS token) is a learnable vector with dimensions identical to the image patch features. In each training batch, it is copied and added to the beginning of the image patch feature sequence, forming a complete input sequence together with all image patch features. The Transformer's attention mechanism primarily focuses on the relationships between local image patches, lacking a natural global information aggregation point. During model training, the CLS token continuously absorbs feature information from all image patches through the attention mechanism, ultimately becoming a global semantic representation of the entire image, which can be directly used for subsequent classification tasks. Furthermore, in image classification scenarios, a fixed-dimensional vector is needed as the basis for final classification. As the first element of the sequence, the output vector of the CLS token can be directly input into the classification head, avoiding the cumbersome additional pooling operation on all image patch features and simplifying the model structure.

[0052] For example, the visual recognition model introduces a learnable classification token (CLS token) to achieve global information aggregation, with the same dimension as the patch embedding. In each batch, the token is copied and added to the beginning of the embedding sequence, as shown in formula (2): (2); in, This represents the concatenated complete feature sequence, which serves as the input to the subsequent Transformer encoder. It contains global information markers and all image patch features; The CLS token represents a learnable classification label, which serves to aggregate global semantic information across the entire image.

[0053] Since the Transformer lacks the ability to model sequence order, Position Embedding needs to add its position information to each token. This module uses learnable position encoding parameters as shown in Equation (3), and the final embedded sequence representation is shown in Equation (4): (3); (4); in, This represents learnable positional encoding features, used to provide spatial location information for each token, compensating for the Transformer's inability to perceive sequence order; express The dimension is N+1, which corresponds to the total number of tokens after concatenation, i.e., N image patch features plus 1 CLS token; The final output image is a sequenced feature representation, which serves as the input for the subsequent Transformer encoder. The token is added element-wise to Z using a broadcast mechanism to inject location information into each token.

[0054] It should be noted that the information obtained above... Overfitting is prevented through Dropout. The image embedding module encodes image patches using convolution, offering a wider receptive field compared to traditional flattening schemes. It also supports configurable overlap strategies, enhancing the information fusion capability of local regions and making it more suitable for FGVC tasks. The introduction of CLS tokens and positional encoding ensures the modeling capability of global information and the ability to represent token order, providing stable input for subsequent Transformer modules.

[0055] Furthermore, to better reproduce the code, this embodiment of the invention provides an example algorithm for converting the original image into a serialized patch token that can be processed by Transformer. The embedding algorithm used is as follows: / / This algorithm converts two-dimensional image data into one-dimensional token data. / / Input: x / / Output: x 1: x = patch_embeddings(x) 2:x = flatten + transpose 3: x = concat(cls_token, x) 4: x = x + position_embeddings 5: x = dropout(x).

[0056] Here, "x=patch_embeddings(x)" performs a convolution operation on the input image, converting it into image patches of fixed size, and extracting local features from each patch to form an initial patch token sequence; "x=flatten+transpose" flattens the two-dimensional features of each patch into a one-dimensional vector and transforms the feature matrix to meet the input requirements of the Transformer, adjusting it to (B, N, D); "x=concat(cls_token,x)" concatenates a trainable cls token at the beginning of the patch token sequence to extract global semantic information of the image; "x=x+position_embeddings" adds positional encoding to the representation of each token, thus preserving the spatial position information of each token in the image, enabling the Transformer to capture the relative positional relationships between tokens in the sequence; and "x=dropout(x)" performs a dropout operation on the input sequence, introducing random dropping to alleviate overfitting and improve the model's generalization and robustness.

[0057] In this embodiment of the invention, the discrete feature sequence without location information is converted into a one-dimensional image serialization feature representation that combines local image features and spatial location information through the above steps, which adapts to the input format requirements of the visual transformer and lays the data foundation for subsequent cross-layer attention extraction and feature encoding.

[0058] S3. Input the image serialization feature representation into the visual transformer of the visual recognition model, and obtain the index matrix through the cross-layer attention extraction mechanism of multiple encoder layers of the visual transformer.

[0059] For example, such as Figure 1 As shown, the multiple encoder layers of the visual transformer (ViT) refer to the stacked multi-layer Transformer encoder units within the ViT. Each encoder layer contains a multi-head self-attention module and a feedforward neural network. This multi-layered stacked structure allows the model to progressively extract local and global features of the image from shallow to deep, ultimately forming a more discriminative semantic representation. Attention extraction specifically refers to the cross-layer attention extraction mechanism. Its role is to capture and locate the image patch location most relevant to the classification label in each encoder layer. For example, in each encoder layer, the attention weight of the classification label on all image patch features is calculated to quantify the contribution of each image patch to the aggregation of global information. Then, the location index of the image patch most relevant to the classification label is determined based on the attention weight, and invalid indexes are avoided through offset correction. Finally, the corrected location indices from all encoder layers are integrated into an index matrix to lock in core regions with high discriminative power at different semantic levels, providing precise locational basis for subsequent selection of cross-level significant feature subsets.

[0060] Furthermore, step S3 specifically includes steps S31 to S33.

[0061] S31. The image serialization feature representation is sequentially input into multiple encoder layers of the visual transformer to extract the attention weights of the classification labels for all image patches using the encoder layers.

[0062] For example, image serialization feature representation This is the final feature sequence output by the image embedding module in step S2. It already contains the CLS token, image patch features, and learnable positional encoding. After inputting into the multi-layer encoder of the visual transformer, each layer encoder calculates attention weights through a multi-head self-attention mechanism, thus providing a foundation for subsequent extraction of cross-layer attention indices. For example, Let be the attention weight matrix of a certain layer in the network. The attention matrix is ​​sliced ​​using the Part_Attention function of TransFG. The specific operation is shown in formula (5): (5); in, This represents the attention weight matrix obtained after slicing, which contains only the attention weights of CLS Tokens for all image patches and is the basis for subsequent localization of key regions. This indicates that all batches should be retained. The slicing operation is represented by the colon ":" in the first dimension, which means to retain all attention heads; the colon ":" in the second dimension, which means to retain all query tokens; the colon "0" in the third dimension, which means to select the CLS Token as the attention weight for the query; and the colon "1" in the fourth dimension, which means to exclude the CLS Token itself and retain the attention weight for other image patches.

[0063] Next, find the location index of the image block that the CLS Token is most interested in, as shown in formula (6): (6); in, This represents the maximum value of the attention weight of the CLS Token for each image patch in LastMap; This indicates the image patch location index corresponding to the maximum value, representing the location of the local region that the CLS Token is most interested in at the current encoder layer; This indicates that the operation of taking the maximum index is performed on dimension 2 (i.e., the image patch dimension).

[0064] The visual model extracts features from different semantic levels from the input image from shallow to deep through stacked multi-layer encoders. It uses the Part Attention mechanism to establish connections between features at different levels, focusing on locating and aggregating key local features across levels, such as fine-grained discrimination regions like bird beaks and petals.

[0065] S32. Determine the location index of the image block that the classification label is most concerned about in each encoder layer according to the attention weight.

[0066] For example, given B is the batch size, H is the number of attention heads, and N is the number of encoder layers involved in finding the maximum index, i.e., the number of layers that actually extract the location index. This represents the total number of encoder layers in the Vision Transformer (ViT). The relationship with N is shown in Equation (7), which means excluding the last layer from all encoder layers and applying it to the preceding layers. The layer extracts attention location indices to obtain local attention information across layers, ultimately yielding the location index of the most concerned image patch. As shown in formula (8): (7); (8); in, Indicates the first The position index matrix obtained from the layer encoder represents the position of the image patch that the CLS Token of that layer is most interested in; This indicates the dimension definition of the index matrix; This indicates the range of index levels retrieved, covering levels from 1 to N. A layer 1 encoder; Let N be the number of Transformer layers in ViT. Each layer extracts the region most relevant to the current Attention and retains its location index. A total of N layers' location indices need to be extracted. For example, in ViT-Base, the number of Transformer layers is... Therefore, it can be extracted from ViT-Base. The layer's location index information is processed by the Feature Reinforcement (FR) module. The attention weights are extracted from the location index of the region of maximum interest in the layer information, and then feature fusion is performed.

[0067] S33. The image block position indices obtained from each encoder layer are offset and corrected. All the corrected image block position indices are then integrated to obtain an index matrix.

[0068] For example, to implement FRM, the image patch position index of the output of the preceding coding layer is first... Position offset correction is performed as shown in formula (9). Then, the maximum attention of layer L-1 after position offset correction is used to construct a composite index matrix, as shown in formula (10). (9); (10); in, Indicates the first The set of raw attention location indices output by the layer encoder; This indicates that an offset correction is performed on the original index, adding 1 to each index value to avoid invalid indexes (index 0) corresponding to the CLS Token, and to ensure that the indexes extracted later only point to valid image blocks; This is the concatenated index matrix; This means concatenating all the offset-corrected attention position indices along the index dimension to form a unified cross-layer index matrix; , which represents the total number of image patches selected in each sample, i.e., the number of indices per sample in the cross-layer index matrix.

[0069] In this embodiment of the invention, by sequentially inputting the image serialization feature representation into multiple encoder layers of the visual transformer, the attention weights of the classification labels on all image patches are extracted layer by layer, and the position index of the most concerned image patch in each layer is located. Then, the index matrix is ​​obtained through offset correction and cross-layer index integration. This process can fully explore the local attention information of the multi-layer encoder layers. Through cross-layer integration, it can effectively filter the local feature noise that is prone to occur in single-layer attention, so that the key area of ​​the localization has stability and discriminability across semantic levels. At the same time, the offset correction operation can accurately avoid invalid indexes on the classification labels themselves, ensuring the accuracy of local area position information. This provides a reliable positional basis for subsequent accurate selection of cross-level high discriminative feature subsets, effectively solving the semantic instability problem caused by relying only on single-layer local features in traditional technologies, and laying a solid foundation for feature localization to improve the accuracy and robustness of fine-grained image recognition.

[0070] S4. Based on the index matrix, filter cross-level feature subsets from the image serialization feature representation.

[0071] For example, cross-level features refer to features jointly selected through an attention mechanism from multiple different encoder layers (rather than a single layer) of the Vision Transformer (ViT). These encoder layers, from shallow to deep, capture low-level details (such as edges and textures) and high-level semantics (such as overall contours and category information) of the image. Cross-level features are the set of salient region features that all encoder layers at these different semantic levels focus on. Fine-grained image recognition relies on subtle local features (such as bird beaks and feather textures), and this key information is often scattered across the encoder outputs of different levels. Cross-level selection can accurately locate the salient regions that all levels focus on, directly locking in the most discriminative core features and avoiding interference from redundant information.

[0072] For example, the input feature sequence (Where B is the batch size, L is the sequence length, and D is the feature dimension) By performing feature selection on the input of the last encoding layer using the index matrix, as shown in formula (11), the aggregated feature tensor can be obtained. , Represents salient regional characteristics across different levels: (11); in, For a single sample, the batch index has a value range of 1. This means that the feature filtering operation is performed independently on the b-th sample in the batch, and the batch processing is completed by traversing all samples. This represents the image serialization feature representation of the b-th sample in the batch, i.e., the final feature sequence after encoding by the image embedding module, adding CLS tokens and position encoding; Let be the index matrix of the b-th sample in the batch; This indicates that an index-based localization and extraction operation is being performed, which refers to the serialization of features from the original image of the b-th sample. In China, accurately extract composite indexes. Features of all element-pointing locations are used to retain only the salient region features that are jointly focused on by cross-layer attention, while irrelevant features are removed; This is the subset of cross-level salient region features obtained from the b-th sample after index extraction. This feature set contains only highly discriminative features jointly selected by multiple encoder layers and multiple attention heads, providing accurate and high-value input for subsequent feature enhancement.

[0073] In this embodiment of the invention, unlike existing feature enhancement methods that directly perform weighted fusion of the complete feature sequence (referring to a global feature sequence containing CLStoken, all image patch features, and positional encoding, which contains a large amount of non-critical redundant information) or a single-layer Transformer output, the enhancement object of FRM in this invention is not the original feature sequence, but a subset of salient region features jointly selected by the attention mechanisms in multiple Transformer Encoder layers. This design focuses on highly discriminative local features that are jointly concerned across layers, rather than indiscriminately enhancing global features. This avoids interference from redundant information and allows the model to more accurately capture the key details required for fine-grained recognition, thereby improving the efficiency and effectiveness of feature enhancement. This feature subset is constructed through the maximum response position index of cross-layer attention, which essentially reflects the consistency of the model's ability to discriminate local regions at different semantic levels. Compared to methods that rely solely on single-layer attention or continuous feature fusion, this design can effectively suppress noise regions introduced by single-layer attention fluctuations, allowing the subsequent feature enhancement process to focus on key regions that maintain discriminative significance throughout the multi-layer encoding process. Therefore, FRM does not perform global reweighting of features, but rather selective feature enhancement based on cross-layer saliency consistency. This mechanism is not a direct derivation by those skilled in the art within the existing feature fusion framework.

[0074] S5. Input the feature subset into the feature enhancement module of the visual recognition model, and perform feature weighted fusion on the feature subset through the feature enhancement module to obtain a fused feature containing global semantics and fine-grained features.

[0075] For example, global semantics refers to the high-level semantic information obtained by aggregating the entire image using classification tokens (CLS tokens). It reflects the overall content and category attributes of the image, such as directly indicating that it is a bird, rather than local details. Fine-grained features refer to highly discriminative local details in the image, which are key to distinguishing similar categories, such as the color of the beak, the texture of the feathers, and the shape of the claws. Fine-grained features are salient region features selected from multiple encoder layers and of common interest across layers, serving as the basis for fine-grained image recognition.

[0076] Further, step S5 specifically includes: inputting the feature subset into the feature enhancement module, performing nonlinear mapping and normalization on the feature subset through the feature enhancement module to obtain the weight coefficients of each feature in the feature subset; performing weighted summation on the feature subset based on the weight coefficients to obtain enhanced fine-grained features; and fusing the fine-grained features with the image serialization feature representation through residual connection to obtain fused features containing global semantic features and fine-grained features.

[0077] For example, in order to further characterize the relative contributions of different salient region features to the final discrimination result, the present invention aggregates the subset of salient region features. Nonlinear mapping and normalization are performed to generate the corresponding weight matrix. In this process, the Dropout layer in the Feature Enhancement Module (FRM) is used to enhance generalization, and the Softmax normalization function ensures that the sum of all weight values ​​is 1. The specific operations of nonlinear mapping and normalization are shown in formulas (12) and (13): (12); (13); Where A represents the intermediate feature matrix after nonlinear transformation; This represents the tangent activation function, used to introduce nonlinear mapping and enhance the expressive power of features; This represents the learnable weight matrix, with dimensions D×D; Represents a learnable bias vector; This represents the normalized weight matrix, which contains all the weight coefficients of the feature subset. Each weight coefficient represents the relative contribution of its corresponding salient region feature to the final result. This represents a learnable weight matrix with dimension D×1, which maps intermediate features to scalar weights. This represents a learnable bias vector.

[0078] Then, the input features of the last original coding layer are enhanced using residual features, based on weights. aggregate feature subsets Weighted fusion is performed, and the input features of the original coding layer are enhanced by scaling the residual connections, as shown in Equation (14): (14); Where F represents the aggregated feature after weighted fusion. The dimension is B×D, which integrates the discriminative information of all salient regions; T represents the number of salient region features contained in a single sample; Let be the weight of the t-th salient region feature in the b-th sample; This represents the t-th salient region feature in the b-th sample.

[0079] Finally, the fused features are broadcast to the original sequence length and the residuals are superimposed. This process satisfies the following: (15); in, The final output of the Feature Enhancement Module (FRM) is an enhanced feature sequence that integrates global semantics and fine-grained features, which serves as the input for subsequent classification tasks. For the serialization of features of the original image ; The representation layer normalization operation is used to stabilize gradient propagation during training. This is a constant used for stable gradient propagation in Layer Norm; This indicates a broadcast expansion operation that expands the dimension of the aggregated feature F from B×D to B×L×D, aligning it with the dimension of the original feature sequence.

[0080] It should be noted that in step S5 above, a subset S of salient region features across multiple encoder layers is selected. These features are the basis for fine-grained recognition and represent the most discriminative local details in the image. Formulas (12) and (13) perform nonlinear mapping and normalization on the feature subset S to generate a weight matrix. This weight characterizes the relative importance of each fine-grained feature to the final judgment. Formula (14) uses the weights Weighted summation of S yields aggregated feature F, which is a condensed representation of all high-value fine-grained features. Since the original image serialization feature representation itself contains global semantic information aggregated from CLS Tokens, as well as the basic features of all image patches, this invention does not directly discard this global information but uses it as the basis for residual connections. Formula (15) is then used... The aggregated fine-grained feature F is broadcast expanded to align its dimensions with the original global feature X, and then connected via residuals. The targeted enhancement of fine-grained features is superimposed onto the original global features. This superposition is not a simple addition, but rather allows the fine-grained features to enhance the parts of the global features most relevant to the discrimination, while preserving the integrity of the global semantics. Finally, the LayerNorm operation ensures the stability of the fused feature distribution, providing optimal input for subsequent classification tasks that includes both global semantics and focuses on key details.

[0081] Furthermore, this embodiment of the invention provides pseudocode for FRM to implement the above process. The main operation flow of FRM is to focus on processing the attention extracted from each layer. An algorithm example is as follows: / / This algorithm is used to fuse all the maximum attention regions before the last layer to enhance attention. / / Input: hidden_states, part_atten_weights / / Output: / / Attention index offset, concatenation and extraction of attention adjusted_indices = [idx + 1 for idx in part_attn_weights] combined_indices = concat(adjusted_indices, dim=1) selected_features = gather hidden_states using combined_indices / / Calculate attention score attn_scores = linear1(selected_features) attn_scores = tanh(attn_scores) attn_scores = dropout(attn_scores) attn_scores = linear2(attn_scores).squeeze(-1) attn_weights = softmax(attn_scores, dim=1) attn_weights = dropout(attn_weights) / / Perform residual join attn_output = weighted_sum(selected_features, attn_weights) attn_output = expand(attn_output, repeat=L) / / Output fusion attention after layer normalization output = LayerNorm(hidden_states + lambda * attn_output).

[0082] For example, in the algorithm above, "adjusted_indices = [idx + 1 for idx in part_attn_weights]" aims to shift the attention index to avoid extracting cls_token; "combined_indices = concat(adjusted_indices, dim=1)" concatenates multiple local indices into a complete index sequence, thus focusing on all tokens; "selected_features = gather hidden_states using combined_indices" is used to collect features from the hidden states using the merged indices; "attn_scores = linear1(selected_features)" feeds the features into the first linear layer; "attn_scores = tanh(attn_scores)" indicates that the tanh function is used for activation; "attn_scores = dropout(attn_scores)" indicates that Dropout is used to prevent overfitting; "attn_scores = "linear2(attn_scores).squeeze(-1)" means feeding the features into a second linear layer to map a scalar attention score and removing the last dimension; "attn_weights = softmax(attn_scores, dim=1)" means normalizing the attention scores using softmax to obtain the weight of each token; "attn_weights = dropout(attn_weights)" means performing a dropout operation on these attention weights to prevent overfitting; "attn_output = weighted_sum(selected_features, attn_weights)" means using the attention weights to perform a weighted sum of these selected features to obtain an augmented feature; "attn_output = expand(attn_output, repeat=L)" means repeatedly expanding the augmented feature of each sample to [B, L, D] to adapt it to the shape of the original sequence. "output = LayerNorm(hidden_states + lambda* attn_output)" means that the enhanced features are fused into the original features according to the λ ratio, and LayerNorm is used for regularization to keep the output dimension consistent with the input dimension.

[0083] In this embodiment of the invention, nonlinear mapping and normalization are used to accurately characterize the discriminative contribution of each significant region feature, enabling targeted enhancement of the weighted summed fine-grained features and highlighting the expression of core discriminative information. Simultaneously, the residual connection fusion method effectively integrates the enhanced fine-grained features while preserving the global semantic integrity of the original features. This avoids the loss of global semantics during feature enhancement and compensates for the lack of fine-grained information in the original features. Furthermore, this module processes only the high-value feature subset selected across layers, rather than operating indiscriminately on global features, improving the targeting and efficiency of feature enhancement. This effectively solves the problems of insufficient fine-grained information and computational redundancy caused by global feature enhancement in traditional techniques. The final output fused features balance the integrity of global semantics and the discriminative power of fine-grained features, providing high-quality feature support for subsequent high-precision fine-grained image classification and significantly improving the classification accuracy and robustness of the visual recognition model.

[0084] S6. Classify and identify the visual data to be identified based on the fusion features.

[0085] For example, after obtaining the enhanced feature sequence that integrates global semantics and fine-grained features, the final classification and recognition can be performed. First, the output vector of the CLS Token is extracted from the enhanced feature sequence. This vector, which has already fused global semantic information and targeted enhanced fine-grained features through residual connections, is the most discriminative representation of the entire image. Then, this vector is input into the classification head of the visual recognition model, and the Softmax function outputs the probability of the image belonging to each category. Finally, the category with the highest probability is selected as the final recognition result. Taking fine-grained bird recognition as an example, global semantic information ensures that the model knows that this is a bird. Enhanced fine-grained features (such as beak shape and feather texture) allow the model to accurately distinguish whether it is a sparrow or a swallow. This fusion method significantly improves the model's recognition accuracy among similar categories.

[0086] In this embodiment of the invention, a standardized preprocessing method is used to unify the input data format. The image embedding module is used to achieve serialized feature encoding of visual data. Then, the cross-layer attention extraction mechanism of the multi-encoder layer of the visual transformer is used to accurately locate key regions of cross-layer interest. A subset of cross-layer features with high discriminative power is selected. Through weighted fusion operations by the feature enhancement module, fine-grained features and global semantic features are organically combined. Finally, classification and recognition are completed based on the fused features. This invention not only accurately captures fine-grained visual features through cross-layer attention extraction, solving the problem of insufficient attention to local key features in traditional recognition methods, but also takes into account the integrity of global semantic information through weighted feature fusion, effectively improving the accuracy and robustness of visual recognition. At the same time, while ensuring recognition performance, the calculation process of feature selection and enhancement is simplified, improving the execution efficiency of the recognition method. It is applicable to various fine-grained visual recognition scenarios.

[0087] Furthermore, the training process of the visual recognition model includes steps S101 to S104.

[0088] S101. Configure training parameters and initialize the network parameters of the visual recognition model; wherein, the training parameters include training execution parameters and optimization strategy parameters, the training execution parameters include the total number of training steps and model validation parameters (such as validation batch size, validation interval steps, etc.), and the optimization strategy parameters include the learning rate and the number of warm-up mechanism steps. The training execution parameters may further include the training batch size.

[0089] For example, the model hyperparameters (including training parameters and network parameters) used in this invention and their corresponding values ​​are shown in Table 3. It should be noted that the hyperparameters involved in Table 3 are only examples, and in practical applications, they can be set to other values ​​according to the actual situation.

[0090] Table 3 Examples of model hyperparameter values

[0091] For example, the pre-trained model used in this invention is ViT-B_16, based on the ViT-Base (basic version of the visual transformer) architecture, employing a 16×16 image patching strategy, and pre-trained on the ImageNet-21k dataset. ImageNet-21k is a public dataset containing 21,000 categories and approximately 14 million images. The input image size is... The size of the image after segmentation is The segmented image has a total of Each of the following tiles is transformed through linear projection: The input sequence is formed by a dimensional vector. The total number of training steps is configured to 10000, representing the total number of global iterations for model training, after which training terminates. The training batch size is configured to 8, representing the number of samples input to the model in each training iteration; this value needs to be adapted to the GPU memory capacity. The validation batch size is configured to 8, representing the number of samples input to the model in each validation phase; maintaining consistency with the training batch size ensures computational stability. The validation interval is configured to 100, meaning validation is performed every 100 training steps to monitor model performance changes on the validation set. The learning rate is configured to 3e-2 (i.e., 0.03), which is the step size for updating model parameters and determines the speed of gradient descent. The learning rate decay method is configured to cosine, indicating the use of a cosine annealing strategy; the learning rate decays non-linearly with the number of training steps according to a cosine function, balancing initial convergence speed with later fine-tuning accuracy. This embodiment of the invention uses a cosine annealing algorithm with a learning rate and sets the warm-up mechanism to 500 steps, meaning that the learning rate linearly increases from 0 to the set value during the first 500 training steps, avoiding model oscillations caused by a large learning rate in the initial stage. For image segmentation, the Overlap method of TransFG is used, and [the following settings are made]. , is the weight coefficient of residual fusion in the Feature Enhancement Module (FRM), which controls the fusion ratio of fine-grained enhanced features and original features, as shown in formula (15). The random seed is configured as 42 to fix the initial state of all random operations and ensure that the experimental results are reproducible. The mixed precision training switch is configured as True, which means that FP16 mixed precision training is enabled, which reduces memory usage and improves training speed through automatic mixed precision technology. The image block strategy is configured as Overlap, which means that an overlapping block strategy is adopted to avoid local feature fragmentation caused by non-overlapping blocks. The sliding step is configured as 12, which means the sliding step when dividing the image (in conjunction with the block size of 16×16), to ensure that there is a 4-pixel overlap area between blocks and to retain more local feature information.

[0092] Furthermore, the learning rate is dynamically adjusted using a cosine annealing scheduling strategy; wherein the cosine annealing scheduling strategy indicates the variation pattern of the learning rate in different warm-up stages.

[0093] For example, to improve the convergence efficiency and performance stability of the model during training, this embodiment of the invention employs a dynamic learning rate scheduling strategy based on cosine annealing. This strategy accelerates convergence by initially using a larger learning rate, and then gradually reduces the learning rate, which helps the model to perform more detailed parameter fine-tuning when approaching the optimal solution, thereby improving the final accuracy. This invention sets the initial learning rate to 3e-2, with a warm-up step count of 500. Therefore, the scheduler linearly increases from a small initial value to the set peak value in the first 500 training steps, and then enters the formal cosine annealing stage. This cosine annealing follows the shape of a cosine function and decays non-linearly until it approaches zero during the convergence stage. This strategy not only provides faster convergence in the early stages of training but also effectively reduces oscillations in later training stages.

[0094] In the training and debugging of deep learning models, real-time monitoring of the training status is crucial for optimizing model performance. Therefore, this invention introduces a visualization monitoring tool, such as Tensor Board, to track changes in various metrics during training, assisting in model tuning and anomaly detection. Tensor Board was initially designed for the TensorFlow framework but is now widely applicable to various deep learning platforms, including PyTorch. Its functions include plotting training and validation loss curves, accuracy trends, learning rate scheduling graphs, parameter distribution histograms, and gradient change heatmaps. Through this visual information, researchers can more intuitively observe the model's convergence state, overfitting trends, and hyperparameter tuning effects during training. During experiments, the visualization interface can be launched via the command line using "tensorboard --logdir=logs --port 6006," and the training progress can be accessed in real-time using a web browser, providing reliable support for subsequent model analysis and comparative experiments.

[0095] See Figure 6 , Figure 6This invention provides a cosine annealing learning rate curve, which includes an initial rapid rise phase and a cosine descent phase. The initial rapid rise phase indicates that at the start of training, the learning rate quickly increases from 0 to a preset maximum value (approximately 0.03). This design allows the model to quickly explore the parameter space in the early stages of training, avoiding getting trapped in local optima. The cosine descent phase indicates that after reaching its peak, the learning rate will slowly decrease according to the cosine function curve until it approaches 0. This smooth descent allows the model to converge more precisely in the later stages of training, avoiding oscillations caused by sudden changes in the learning rate. Furthermore, when the learning rate reaches its minimum point, it will quickly rise back to its maximum value, starting a new cycle. This periodic restart helps the model escape local optima, further improving the final convergence accuracy.

[0096] S102. Obtain the visual recognition dataset and divide the visual recognition dataset into training set data and test set data.

[0097] For example, in the FGVC domain, datasets are typically used to classify subcategories with high similarity. CUB-200-2011 is commonly used in this domain, and this embodiment of the invention also uses the CUB-200-2011 dataset for validation. This dataset is about birds, with 200 subcategories of birds and a total of 11,788 images. See [link to documentation]. Figure 7 , Figure 7 This is a schematic diagram of a dataset subclass sample provided in an embodiment of the present invention. Figure 7 The first two rows and the third and fourth rows represent the same category. Both sets of categories showcase bird data from different species, with slight differences in image state, background, and shooting angle. This realistically reflects the diversity of samples obtained from field photography. For example, several categories show small songbirds, typically with bright plumage and slender beaks; while other categories include larger birds or birds in specific habitats, clearly demonstrating significant differences in body structure and plumage among species. From a visual perspective, the diversity presented in these images also presents a challenge for the model's recognition. The images include samples with simple backgrounds and good lighting, as well as images with complex backgrounds and occlusions. These images closely resemble the environments faced by image recognition in real-world scenarios. Therefore, the CUB-200-2011 dataset not only provides rich category divisions at the species level but also possesses high image quality and sample complexity, making it suitable as an important basis for evaluating the performance of the FGVC model.

[0098] In this embodiment of the invention, the CUB-200-2011 dataset was preprocessed and loaded using the PyTorch framework. Different data augmentation strategies were designed for the training and test sets to improve the model's generalization ability and classification performance. For the training set, bilinear interpolation was first used to adjust the input image to 600×600 pixels; then, it was randomly cropped to a 448×448 region; next, it was horizontally flipped with a 50% probability to enhance the model's robustness to changes in left and right structure; finally, the image was converted into a tensor and standardized using the mean and standard deviation commonly used in the Image Net dataset, such as mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]. A more conservative processing strategy was used for the test set, similarly scaling the image to 600×600 and then cropping it to 448×448 to avoid introducing randomness and ensure the stability and feasibility of the test results.

[0099] S103. Input the training set data into the initialized visual recognition model, obtain the fusion features output by the visual recognition model, and obtain the sample prediction category based on the fusion features.

[0100] For example, training data labeled with real category information is input into a visual recognition model that has not been trained and whose network parameters are initially random values. The model will execute the complete inference process of steps S2 to S5 above. First, the input data is divided into image blocks, and CLS tokens and position encoding are added to obtain image serialization feature representation. Then, the cross-layer attention mechanism of the multi-layer encoder of the visual transformer is used to filter the feature subset of cross-layer salient regions. The feature enhancement module completes the weighted fusion of fine-grained features and global semantic features, and finally outputs the fused features. At the same time, the model uses the fused features to access the classification head for calculation to obtain the sample prediction category corresponding to each training sample, that is, the model's judgment result of the category to which the sample belongs.

[0101] S104. Calculate the loss value based on the predicted category and the true category of the sample, and update the network parameters of the visual recognition model based on the loss value until the preset number of training steps is completed or the model verification accuracy converges, and obtain the trained visual recognition model.

[0102] For example, the predicted sample category obtained in step S103 and the true category (accurate category information manually labeled) inherent in the training set data are substituted into the preset loss function, such as... Figure 1The cross-entropy loss function in the model calculates the loss value by measuring the difference between the two values. The larger the loss value, the greater the deviation between the model's current prediction and the actual situation, and the more urgent the need for network parameter optimization. Subsequently, the backpropagation algorithm is used to pass the loss value from the model's output layer to the input layer layer by layer. Based on the gradient information of the parameters of each layer, all learnable network parameters of the visual recognition model (including attention layer weights, W1 / W2 of the feature enhancement module, bias vectors b1 / b2, etc.) are iteratively updated according to a preset optimizer strategy (such as Adam, SGD) until the preset number of training steps is completed (such as a fixed training of 10,000 steps) or the model's recognition accuracy on an independent validation set tends to stabilize (i.e., the validation accuracy no longer changes significantly with the increase of training steps, and the model converges). Training is then stopped, resulting in a visual recognition model that has been trained and whose network parameters have been optimized to the optimal state, which can be used for classification and recognition of subsequent visual data.

[0103] Furthermore, this embodiment of the invention provides a three-way performance comparison process using ViT as the baseline, TransFG as the intermediate comparison model, and PAFR-ViT as the model proposed in this invention, comparing the training accuracy and loss changes of the three in fine-grained image recognition tasks.

[0104] See Figure 8 , Figure 8 This is a comparison chart of the model accuracy of the three models provided in this embodiment of the invention after 10,000 training steps. As the number of training steps increases, the accuracy on the test set increases significantly in all models, and then tends to stabilize, reflecting that all three models correctly capture the differences between subcategories. Observing the three training curves, it can be analyzed that ViT is in a rapid feature building phase from 0 to 4000 steps, and enters a gradual convergence phase from 4000 to 10000 steps; TransFG and PAFR-ViT are both in a rapid feature building phase from 0 to 2000 steps, and enter a gradual convergence phase from 2000 to 10000 steps. Furthermore, all three models converge from 8000 to 10000 steps, with relatively small changes in accuracy. During the model convergence phase, it can be analyzed that PAFR-ViT's accuracy is 0.01% lower than TransFG's, but both PAFR-ViT and TransFG have higher accuracy than ViT.

[0105] See Figure 9 , Figure 9This is a comparison chart of the model losses of the three models provided in this embodiment of the invention after 2500 training steps. The chart compares the loss changes of the three models in the first 2500 training steps. Overall, all three loss curves tend to converge around the 1700th step, indicating that the models have completed the main parameter adjustment process in the early stages of training. In addition, from the trend of loss decrease, it can be observed that there are significant differences in convergence speed and stability among the different models. Among them, ViT has the most drastic loss fluctuations in the early stages of training, showing that its parameter adjustment is relatively unstable in the early training stage; the curve of TransFG is slightly more stable than that of ViT, but still has some oscillations. In contrast, the PAFR-ViT model proposed in this invention performs better in terms of loss decrease speed and curve smoothness, showing faster convergence ability and higher training stability. This result further verifies the potential of the PAFR-ViT architecture in optimizing convergence speed and model stability, and is expected to achieve more efficient model training in fine-grained classification tasks.

[0106] The highest accuracy achieved by the model on the test set during the model training process is shown in Table 4. Among them, TransFG (Transformer-based Fine-Grained) is a baseline model for fine-grained image classification using the Transformer architecture; FFVT (Feature Fusion Vision Transformer) is an improved ViT model that enhances fine-grained recognition performance through multi-feature fusion; PAFR-ViT-TransFG (Part-Aware Feature Reinforcement Vision Transformer on TransFG) is an improved model obtained by integrating the PAFR module proposed in this invention into the TransFG baseline model; PAFR-ViT-FFVT (Part-Aware Feature Reinforcement Vision Transformer on FFVT) is an improved model obtained by integrating the PAFR module proposed in this invention into the FFVT baseline model. Accuracy refers to the proportion of samples correctly classified by the model on the test set out of the total samples, reflecting the model's recognition performance; Training Time is the training time, which is the time (in hours) taken for the model to complete the entire training process, reflecting the training efficiency of the model; ATT (Accuracy per Training Time) refers to the ratio of the accuracy improvement or final accuracy obtained by the model per hour of training time to the training time. It is an indicator that comprehensively measures the model performance and training efficiency. The higher the value, the higher the cost-effectiveness of the model.

[0107] Table 4 Performance of different methods on the CUB-200-2011 dataset

[0108] Table 4 shows that the ATT (Average Time Tolerance) metric more intuitively reflects the accuracy gain of the model per unit time. Although TransFG achieves slightly higher accuracy due to its complex attention mechanism, its ATT is 59.53, while PAFR-ViT's is 116.67. TransFG's mechanism of multiplying the attention matrices of each layer and selecting the region with the maximum value significantly increases training time and affects overall efficiency. In contrast, PAFR-ViT, while maintaining comparable accuracy to TransFG, effectively improves training efficiency and comprehensive utilization of attention regions by selecting the most relevant attention regions in each layer and fusing them in the last layer. This results in more practically valuable performance, with its ATT far exceeding that of the classic fine-grained recognition model TransFG, demonstrating a superior time-performance balance.

[0109] Furthermore, to explore in depth To assess the effect of this invention on model performance, a parameter sensitivity analysis was conducted on the CUB-200-2011 dataset. The analysis results are shown in Table 5.

[0110] Table 5 shows the differences in PAFR-ViT across the CUB-200-2011 dataset. The result of the value

[0111] As shown in Table 5, when When the value varies within the range of 0.2 to 2.0, the model does not change accordingly. The value increases and changes monotonically; when At that time, the model's test accuracy reached its peak, with a test accuracy of 91.17%; compared to the baseline setting. At that time, the test accuracy was 91.05%, an improvement of 0.12 percentage points. This experiment shows that appropriate regularization constraints can improve the performance of fine-grained classification. However, At this point, the test accuracy actually decreased to 90.82%, indicating that excessive regularization can suppress discriminative features. It is worth noting in the experiment that when... At that time, the test accuracy rebounded to 91.00%. Regarding training time, the time required for the model varied little across the different experimental groups, remaining around 1.47 hours, indicating... Changes in accuracy have a relatively small impact on training costs. Therefore, in practical applications, adjustments can be made flexibly based on accuracy performance. The value is used to achieve a better balance between accuracy and training stability.

[0112] Furthermore, in order to fully verify the comprehensive performance advantages of the method of the present invention in fine-grained image recognition tasks, the present invention also studied the performance of PAFR-ViT and other cutting-edge algorithms on the CUB-200-2011 dataset. The analysis results can be found in Table 6.

[0113] Table 6. Performance comparison with other cutting-edge algorithms on the CUB-200-2011 dataset.

[0114] As can be seen from Table 6, the ATT value of PAFR-ViT-TransFG (116.496) is significantly higher than other cutting-edge methods: it is 1.69 times that of FAL-ViT (69.074), 1.58 times that of SIM-Trans (73.640), and 1.59 times that of SM-ViT (73.325). This significant difference directly confirms that PAFR-ViT can provide high accuracy at a lower time cost.

[0115] Furthermore, this invention analyzes the visualization effects of PAFR-ViT, ViT, and TransFG models on bird image attention in the FGVC task using Grad-CAM (Gradient-weighted Class Activation Mapping). Grad-CAM is a gradient-based class activation mapping technique commonly used for visual analysis of model decision-making in deep neural networks. This technique calculates gradients of the output feature maps of specific convolutional layers for the target category and weights these gradients to obtain a heatmap. By overlaying the Grad-CAM heatmap onto the original image, the distribution of image regions that the model focuses on during classification can be visually observed. Significant differences exist in the attention values ​​of different models when perceiving key information regions of the image. Introducing Grad-CAM improves the interpretability of the model and provides a visual basis for further optimization of attention and network architecture.

[0116] See Figure 10 , Figure 10These are attention visualizations of the three models provided in this embodiment of the invention. PAFR-ViT exhibits a more concentrated attention distribution across all images, primarily focusing on key areas of the bird's body, such as the beak, head, and wings. This demonstrates the effectiveness of PAFR-ViT's local feature extraction and region attention mechanism. In contrast, the ViT model's attention distribution is more dispersed, sometimes even focusing on the background while neglecting the target subject. While the TransFG model demonstrates good focusing ability on local bird regions in some images, its attention distribution remains relatively dispersed compared to PAFR-ViT, with some attention not fully focused on discriminative features. The effectiveness of the PAFR-ViT model in the FGVC task is verified using Grad-CAM.

[0117] Compared to existing technologies, the visual recognition method disclosed in this invention significantly enhances the model's ability to model fine-grained discriminative features by reorganizing the attention mechanism and feature flow pattern within the Transformer, without significantly increasing the model's parameter size or training complexity. This method has significant implications for training large multimodal models.

[0118] This invention does not merely utilize attention information from a single Transformer Encoder layer. Instead, it extracts the location indices of the maximum response regions corresponding to the attention mechanisms from multiple encoding layers, using these indices as a crucial basis for subsequent feature modeling. Unlike existing technologies that commonly employ single-layer attention analysis or rely solely on the attention output of the last layer, this invention extracts attention response regions across layers. This allows the model to explicitly capture key regions that are repeatedly focused on at different semantic levels, thereby characterizing the stable discriminative ability of local regions during semantic evolution. Through this approach, the invention effectively alleviates the problems of single-layer attention being susceptible to noise interference and unstable regions of focus, improving the model's reliability in locating fine-grained key regions and providing a more stable saliency foundation for subsequent feature fusion and enhancement.

[0119] Furthermore, building upon cross-level attention extraction, this invention proposes a hierarchical feature fusion strategy based on saliency consistency. Utilizing discrete location indices obtained from cross-level attention extraction, the original feature sequence is inversely indexed, selecting only local region features commonly emphasized across multiple Transformer Encoder layers, and constructing a cross-level salient region feature set. Unlike existing techniques that directly fuse complete feature sequences or continuous feature maps, this invention uses an index-driven feature selection mechanism to limit the feature fusion process to key regions with cross-level semantic consistency, thereby avoiding interference from irrelevant regions or noisy features. This hierarchical feature fusion method enables the model to maintain global structural information while focusing on enhancing the modeling ability of fine-grained discriminative regions, significantly improving the discriminativeness of feature representation in complex backgrounds or scenarios with small inter-class differences.

[0120] Based on the aforementioned cross-layer attention extraction and hierarchical feature fusion results, this invention proposes a Feature Enhancement Module (FRM). Instead of globally reweighting the original feature sequence, FRM adaptively characterizes the discriminative contribution of different salient regions through learnable nonlinear mapping and weight modeling for a subset of salient region features obtained by cross-layer attention consistency filtering. This residual is then applied to the input features of the final Transformer Encoder layer. Unlike existing schemes based on intermediate layer feature enhancement or output feature concatenation, FRM performs targeted feature enhancement during the critical semantic modeling stage before final discrimination, effectively injecting discriminative information without disrupting the original Transformer encoding structure. Through this feature enhancement module, this invention significantly enhances the model's ability to express fine-grained discriminative features while maintaining model training efficiency and structural stability, improving the overall robustness and generalization performance of the model in complex visual tasks.

[0121] See Figure 11 , Figure 11 This is a structural block diagram of a visual recognition device 100 provided in an embodiment of the present invention. The visual recognition device 100 includes: The data preprocessing module 11 is used to acquire visual data to be recognized and to preprocess the visual data to be recognized to obtain preprocessed input data. Encoding module 12 is used to encode the input data through the image embedding module of the visual recognition model to obtain an image serialization feature representation; The position index generation module 13 is used to input the image serialization feature representation into the visual transformer of the visual recognition model, and obtain the index matrix through the cross-layer attention extraction mechanism of multiple encoder layers of the visual transformer; Feature subset generation module 14 is used to filter cross-level feature subsets from the image serialization feature representation based on the index matrix; The feature generation module 15 is used to input the feature subset into the feature enhancement module of the visual recognition model, and perform feature weighted fusion on the feature subset through the feature enhancement module to obtain a fused feature containing global semantics and fine-grained features; The classification and recognition module 16 is used to classify and recognize the visual data to be recognized based on the fused features.

[0122] It is worth noting that the working process of each module in the visual recognition device 100 described in the embodiments of the present invention can refer to the working process of the visual recognition method described in the above embodiments, and will not be repeated here.

[0123] See Figure 12 , Figure 12 This is a structural block diagram of a visual recognition device 200 provided in an embodiment of the present invention. The visual recognition device 200 includes a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, it implements the steps in the various visual recognition method embodiments described above.

[0124] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the visual recognition device 200.

[0125] The visual recognition device 200 may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the schematic diagram is merely an example of the visual recognition device 200 and does not constitute a limitation on the visual recognition device 200. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the visual recognition device 200 may also include input / output devices, network access devices, buses, etc.

[0126] The processor 21 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 21 is the control center of the vision recognition device 200, connecting all parts of the vision recognition device 200 via various interfaces and lines.

[0127] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements various functions of the visual recognition device 200 by running or executing the computer programs and / or modules stored in the memory 22 and calling the data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0128] Wherein, if the modules / units integrated in the visual recognition device 200 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 21, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0129] Furthermore, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the visual recognition method as described in any of the above embodiments.

[0130] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A visual recognition method, characterized by, include: Acquire the visual data to be identified, and preprocess the visual data to be identified to obtain preprocessed input data; The input data is encoded by the image embedding module of the visual recognition model to obtain an image serialization feature representation; The image serialization feature representation is input into the visual transformer of the visual recognition model, and an index matrix is ​​obtained through the cross-layer attention extraction mechanism of multiple encoder layers of the visual transformer; Based on the index matrix, a cross-level feature subset is selected from the image serialization feature representation; The feature subset is input into the feature enhancement module of the visual recognition model, and the feature enhancement module performs feature weighted fusion on the feature subset to obtain a fused feature that includes global semantics and fine-grained features; The visual data to be identified is classified and recognized based on the fusion features.

2. The visual recognition method as described in claim 1, characterized in that, The image embedding module of the visual recognition model encodes the input data to obtain an image serialization feature representation, including: The input data is processed into image blocks to obtain multiple image block features; A classification label is concatenated before the sequence composed of multiple image patch features, and positional encoding features are added to the classification label and each image patch feature to obtain an image serialization feature representation.

3. The visual recognition method as described in claim 1, characterized in that, The cross-layer attention extraction mechanism through multiple encoder layers of the visual transformer yields an index matrix, including: The image serialization feature representation is sequentially input into multiple encoder layers of the visual transformer to extract the attention weights of the classification labels for all image patches using the encoder layers; The location index of the image patch most concerned by the classification label in each encoder layer is determined based on the attention weights. The image block position indices obtained from each encoder layer are offset-corrected, and all the corrected image block position indices are integrated to obtain an index matrix.

4. The visual recognition method as described in claim 1, characterized in that, The step of performing feature weighting and fusion on the feature subset through the feature enhancement module to obtain fused features containing global semantics and fine-grained features includes: The feature subset is input to the feature enhancement module, which performs nonlinear mapping and normalization on the feature subset to obtain the weight coefficient of each feature in the feature subset. The feature subset is weighted and summed based on the weight coefficients to obtain enhanced fine-grained features. The fine-grained features are fused with the image serialization feature representation through residual connections to obtain fused features that include global semantic features and fine-grained features.

5. The visual recognition method as described in claim 1, characterized in that, The training process of the visual recognition model includes: Configure training parameters and initialize the network parameters of the visual recognition model; Obtain a visual recognition dataset and divide the visual recognition dataset into training set data and test set data; The training set data is input into the initialized visual recognition model, and the fusion features output by the visual recognition model are obtained, and the sample prediction category is obtained based on the fusion features; The loss value is calculated based on the predicted category and the true category of the sample. The network parameters of the visual recognition model are updated based on the loss value until the preset number of training steps is completed or the model verification accuracy converges, thus obtaining the trained visual recognition model.

6. The visual recognition method as described in claim 5, characterized in that, The training parameters include training execution parameters and optimization strategy parameters; wherein, the training execution parameters include the total number of training steps and model validation parameters, and the optimization strategy parameters include the learning rate and the number of warm-up mechanism steps.

7. The visual recognition method as described in claim 6, characterized in that, The learning rate is dynamically adjusted using a cosine annealing scheduling strategy; wherein the cosine annealing scheduling strategy indicates the variation pattern of the learning rate in different warm-up stages.

8. A visual recognition device, characterized in that, include: The data preprocessing module is used to acquire visual data to be recognized and to preprocess the visual data to be recognized to obtain preprocessed input data. The encoding module is used to encode the input data through the image embedding module of the visual recognition model to obtain an image serialization feature representation; The location index generation module is used to input the image serialization feature representation into the visual transformer of the visual recognition model, and obtain the index matrix through the cross-layer attention extraction mechanism of multiple encoder layers of the visual transformer; The feature subset generation module is used to filter cross-level feature subsets from the image serialization feature representation based on the index matrix; The feature generation module is used to input the feature subset into the feature enhancement module of the visual recognition model, and perform feature weighted fusion on the feature subset through the feature enhancement module to obtain fused features containing global semantics and fine-grained features; The classification and recognition module is used to classify and recognize the visual data to be recognized based on the fused features.

9. A visual recognition device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the visual recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the visual recognition method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the visual recognition method as described in any one of claims 1 to 7.

Images