The invention provides a ViT model improvement method,
system and device and a storage medium, and belongs to the technical field of
computer vision. The method comprises the following steps: firstly, dividing an input image into a plurality of non-overlapped patches, and carrying out embedding
processing on each patch to generate initial feature sequence representation; clustering the features of the adjacent patches by using a K-means clustering
algorithm with spatial constraint, combining the adjacent patches with similar semantic features into K classes, and extracting the feature of each clustering center as a new feature representation; and finally, inputting the clustered feature sequence into ViT for self-attention calculation and feedforward
network processing, and generating final image feature representation. According to the method, semantic similar patches are aggregated, the length of an input sequence is reduced, the calculation complexity is reduced, redundant information is reduced, the local semantic understanding ability is enhanced, and the method is suitable for visual tasks such as image classification and target detection.