Double-layer topology center clustering method and system

Abstract

The invention discloses a double-layer topology center clustering method and system, and is applied to the technical field of image classification, and the method comprises the steps: obtaining an original image, carrying out the data enhancement of the original image, and obtaining a first view and a second view; respectively inputting the first view and the second view into a classification model for processing, and outputting multi-scale features; wherein the classification model is composed of an encoder, a low-level feature slot module, a high-level feature slot module, a reconstruction module and a comparative learning module; the multi-scale features are clustered, and a clustering result is obtained; and based on the obtained real label set, generating an optimal matching result of the clustering result and the real labels.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and in particular to a double-layer topological center clustering method and system. Background Art

[0002] Image classification is one of the basic tasks in computer vision, which aims to classify input images into predefined categories. Generalized Category Discovery (GCD) expands the scope of this task and requires the identification of both known and unknown categories in a partially annotated image dataset. Unlike traditional semi-supervised learning, GCD requires the model to have the ability to distinguish unknown categories, which places higher requirements on the generalization ability of the model.

[0003] Topological structure refers to the spatial and semantic relationship between the parts of an image. For example, the topological structure of a bird may include the layout of its head, wings, and tail. Through topological center clustering, associations can be established between local and global features of an image, thereby more comprehensively representing image information. Existing studies have shown that topological structure-based learning can effectively improve the classification performance of models on unlabeled data, but the design and implementation of related methods still face great challenges.

[0004] In order to overcome these defects, the present application proposes a double-layer topological center clustering method and system. Summary of the invention

[0005] The purpose of this application is to provide a two-layer topological center clustering method and system to solve the above-mentioned problems.

[0006] To achieve the above objectives, this application provides the following technical solutions:

[0007] In a first aspect, the present application provides a two-layer topological center clustering method, comprising:

[0008] Acquire an original image, and perform data enhancement on the original image to obtain a first view and a second view;

[0009] Inputting the first view and the second view into a classification model for processing respectively, and outputting multi-scale features; wherein the classification model is composed of an encoder, a low-level feature slot module, a high-level feature slot module, a reconstruction module, and a contrastive learning module;

[0010] The multi-scale features are clustered to obtain a clustering result; based on the acquired real label set, a best matching result between the clustering result and the real label is generated.

[0011] In a second aspect, the present application proposes a two-layer topological center clustering system, comprising:

[0012] An acquisition module: acquires an original image, performs data enhancement on the original image, and obtains a first view and a second view;

[0013] Processing module: input the first view and the second view into a classification model for processing, and output multi-scale features; wherein the classification model is composed of an encoder, a low-level feature slot module, a high-level feature slot module, a reconstruction module, and a contrastive learning module;

[0014] Clustering module: clustering the multi-scale features to obtain clustering results; based on the acquired real label set, generating the best matching result between the clustering results and the real labels.

[0015] In a third aspect, the present application provides a device comprising a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing a two-layer topological center clustering method; and the processor is used to execute the program instructions stored in the memory to implement a two-layer topological center clustering method.

[0016] In a fourth aspect, the present application provides a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute a two-layer topological center clustering method.

[0017] The present application provides a two-layer topology center clustering method and system, which has the following beneficial effects:

[0018] (1) Through modeling of low-level feature slot modules and high-level feature slot modules, the detailed features and global semantic features of the image are extracted respectively; this two-layer mechanism significantly improves the performance of the classification model in processing known categories and discovering unknown categories, especially in fine-grained classification tasks.

[0019] (2) The topological contrastive learning framework proposed in this application makes the same semantic features consistent in different views through the coordinated optimization of high- and low-level slots, suppresses pseudo-label noise, improves the generalization ability of the classification model for unlabeled data, and shows excellent recognition accuracy in generalized category discovery tasks;

[0020] (3) This application proposes a multi-scale feature fusion strategy, which generates a unified image representation, namely, multi-scale features, by fusing local features, global features and slot-level features. This strategy is applicable to different scenarios and can achieve higher classification accuracy in complex data distributions. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of a process of a double-layer topological center clustering method according to Example 1 of the present application;

[0022] Figure 2A schematic diagram showing a comparison between a traditional method of Example 1 of the present application and a double-layer topological center clustering method;

[0023] Figure 3 A schematic diagram of a framework of a double-layer topological center clustering method according to Example 1 of the present application;

[0024] Figure 4 A schematic diagram of the structure of a double-layer topological center clustering system of Example 2 of the present application;

[0025] Figure 5 This is a schematic diagram of the device structure of Example 3 of the present application;

[0026] Figure 6 This is a schematic diagram of the storage medium structure of Example 4 of the present application. DETAILED DESCRIPTION

[0027] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0028] The following is an analysis of the solutions in the prior art in combination with related technologies.

[0029] (1) Strong dependence on global features and insufficient utilization of local information:

[0030] Existing GCD methods, such as SimGCD and CMS, mainly rely on global category features to classify unlabeled images through global feature representation. However, important information in images is often contained in local features, such as texture, color, edges, etc. These local features are particularly important for fine-grained category distinction. Ignoring local topological features will make it difficult for the model to effectively distinguish fine-grained categories, especially categories with similar visual features but different semantics.

[0031] (2) The quality of pseudo-label generation is limited:

[0032] Existing methods usually generate pseudo labels through clustering or contrastive learning. For example, DCCL uses InfoMap clustering to generate pseudo labels. However, it is easily affected by noise and category imbalance when dealing with complex image distribution, resulting in a decrease in the quality of pseudo labels. This quality problem will be further transmitted to the subsequent training stage, affecting the model performance.

[0033] (3) Lack of modeling of image topology:

[0034] Images contain rich spatial and semantic topological structures, such as the relationship between a bird's head and wings. Existing methods mainly rely on global features and data augmentation to improve generalization capabilities, but fail to explicitly model these topological information, making it difficult for the model to capture the key components and hierarchical structures of the object.

[0035] (4) Insufficient generalization ability of category expansion:

[0036] The GCD task needs to be expanded to unknown categories based on known categories, but existing methods often rely on limited global features to learn unknown categories and cannot fully utilize the potential semantic structure in unlabeled data, resulting in insufficient recognition ability for unknown categories.

[0037] In response to the above shortcomings, this application introduces a two-layer Slot Attention mechanism to achieve collaborative modeling of local features and global features, thereby improving the model's ability to distinguish fine-grained categories; through topological center contrast learning, feature grouping and clustering of unlabeled data are performed, thereby improving the reliability of pseudo-labels and reducing noise interference; therefore, a two-layer topological center clustering method and system proposed in this application solves the core problems of the prior art in GCD tasks.

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0039] Example 1

[0040] See also Figure 1 , is a flow chart of a double-layer topological center clustering method of Example 1 of the present application; the steps include:

[0041] S1: Acquire an original image, perform data enhancement on the original image, and obtain a first view and a second view.

[0042] In this embodiment, data enhancement is performed on the original image to generate a first view View1 and a second view View2. Views with different visual features are generated by enhancement means such as rotation, cropping, and color conversion, thereby improving the robustness and generalization ability of the classification model.

[0043] S2: Input the first view and the second view into a classification model for processing respectively, and output multi-scale features; wherein the classification model is composed of an encoder, a low-level feature slot module, a high-level feature slot module, a reconstruction module, and a contrastive learning module.

[0044] In this embodiment, the first view and the second view are respectively input into a classification model for processing, which specifically includes:

[0045] Feature extraction: The first view and the second view are input into the encoder respectively, and the local features h of the first view are output.1 , global feature g 1 , and the local feature h of the second view 2 , global feature g 2 The encoder extracts local detail features and overall semantic information in the image as the basis for subsequent feature modeling.

[0046] Initialize low-level slots: Initialize the local features h output by the encoder 1 , the local feature h 2 Input low-level feature slot module S respectively l , output low-level features and The low-level features of the image, such as texture, color, and edges, are captured through the low-level feature slot module, emphasizing view-specific detail information.

[0047] High-Level Slots initialization: Initialize shared high-level feature slots It is used for high-level feature modeling of two views. The local features h output by the encoder are 1 , the local feature h 2 Input high-level feature slot module S respectively h , output high-level features and The high-level feature slot module captures view-independent high-level semantic features such as shape, contour, and topological layout to improve semantic consistency.

[0048] Reconstruction module: The reconstruction module also includes a decoder d, which converts the low-level features output by the low-level feature slot module and Input the decoder and reconstruct it into local feature h l1 and h l2 .

[0049] The low-level feature slot module is optimized by reconstructing the loss function; the expression is:

[0050] L rec1 =||h l1 -h l ||,

[0051]

[0052] Among them, h l1 、h l2 is the reconstructed local feature, h l is the true value, which is the original local feature; L rec1 The reconstruction loss of the first view; L rec2is the reconstruction loss of the second view.

[0053] Contrastive learning module: The contrastive learning module includes a contrastive learning loss function, which is used to perform view alignment; the contrastive learning loss function L SC It is expressed as:

[0054]

[0055] Where τ is the temperature coefficient; K is the number of high-level feature slots; k is the current slot index, which is used to traverse K slots when averaging the loss; k′ is the traversal index when summing the denominator, which is used to convert All slots in the same view Perform similarity comparison and obtain a softmax distribution; The high-level features of the first view indexed by the current slot; is the high-level feature of the second view indexed by the current slot. Using the contrastive learning loss function L SC Perform multi-view alignment to make them consistent in semantic space.

[0056] Based on the above global feature g 1 , global feature g 2 , low-level features Low-level features High-level features High-level features Perform feature pooling and fusion to generate the multi-scale features. The multi-scale features include local information and global information, providing high-quality feature representation for subsequent clustering.

[0057] Furthermore, the classification model also includes: using image scale contrast learning loss function L IC , unsupervised contrastive learning And supervised contrastive learning The multi-scale features are optimized; wherein the formula is expressed as:

[0058]

[0059]

[0060] in, is the global feature of the first view of the i-th sample; is the global feature of the second view of the i-th sample; g n is the feature representation of the nth sample in the first view or the second view. When summing ∑n in the denominator, n traverses other samples except i; τ is the temperature coefficient, which is used to control the "smoothness" of the softmax distribution in the contrast loss; 1 [n≠i]is the indicator function, which is 1 when n≠i and 0 otherwise; it ensures that the current sample itself is not included when calculating the contrast distribution of the denominator; is the index set of other samples with the same label as i. When the sample has a label, Select positive samples with the same label as i; The number of elements in is the number of other samples with the same label as i; B is the set of all sample indexes; is a labeled subset; λ is a weight coefficient used to control the unsupervised contrastive learning term and supervised contrastive learning Based on the above loss function, the classification model’s ability to distinguish different categories is further enhanced, while the consistency and generalization performance of feature representation are improved.

[0061] Through step S2, the performance and applicability of the generalized category discovery task are effectively improved by combining local feature and global feature modeling, topological information comparative learning, and multi-scale feature fusion.

[0062] S3: Clustering the multi-scale features to obtain clustering results; based on the acquired real label set, generating the best matching result between the clustering results and the real labels.

[0063] In this embodiment, the multi-scale features are clustered by k-means, and all samples are divided into several clusters.

[0064] In order to distribute the consistency between the clustering results and the true labels, the best matching result is selected between the clustering cluster index set and the true label set according to the Hungarian algorithm, specifically:

[0065] If the number of clusters is greater than the number of true labels, the extra clusters are assigned to the empty set, and all samples in the clusters are defined as prediction errors; if the number of clusters is less than the number of true labels, the extra labels are assigned to the empty set, and all samples corresponding to the labels are defined as prediction errors. Through this process, the best matching result between the clustering result and the true label is finally obtained.

[0066] In summary, Example 1 of the present application solves the deficiencies of the prior art in terms of global and local feature integration, pseudo-label quality, and topological structure utilization. The present application first performs data enhancement on the original image, extracts local and global features, and initializes modeling through low-level and high-level feature slots. Through the double-layer Slot Attention mechanism, the collaborative modeling of local and global features is realized, which significantly improves the classification model's ability to distinguish fine-grained categories. The reconstruction module and contrastive learning module are then used to enhance the consistency and robustness of the features, and multi-scale features are generated through a multi-scale feature fusion strategy. Finally, feature representation is optimized through image scale contrast learning to achieve efficient classification. The combination of local and global feature modeling, topological information contrast learning, and multi-scale feature fusion effectively improves the performance and applicability of generalized category discovery tasks, and provides new ideas for the technical development of related fields.

[0067] Example 2

[0068] See also Figure 4 , is a schematic diagram of the structure of a double-layer topological center clustering system of Example 2 of the present application; the specific contents include:

[0069] An acquisition module: acquires an original image, performs data enhancement on the original image, and obtains a first view and a second view;

[0070] Processing module: input the first view and the second view into a classification model for processing, and output multi-scale features; wherein the classification model is composed of an encoder, a low-level feature slot module, a high-level feature slot module, a reconstruction module, and a contrastive learning module;

[0071] Clustering module: clustering the multi-scale features to obtain clustering results; based on the acquired real label set, generating the best matching result between the clustering results and the real labels.

[0072] Example 3

[0073] See also Figure 5 , is a schematic diagram of the device structure of Embodiment 3 of the present application. The device 50 includes a processor 51 and a memory 52 coupled to the processor 51 .

[0074] The memory 52 stores program instructions for implementing the above-mentioned double-layer topology center clustering method.

[0075] The processor 51 is used to execute the program instructions stored in the memory 52 to implement a double-layer topological center clustering.

[0076] The processor 51 may also be referred to as a CPU (Central Processing Unit).

[0077] The processor 51 may be an integrated circuit chip with signal processing capabilities. The processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0078] Example 4

[0079] See also Figure 6 , which is a schematic diagram of the structure of the storage medium of Example 4 of the present application. The storage medium of the embodiment of the present application stores a program file 61 that can implement all the above methods, wherein the program file 61 can be stored in the above storage medium in the form of a software product, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a computer, a server, a mobile phone, a tablet, and other devices.

[0080] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0081] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

[0082] Although the embodiments of the present application have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the appended claims and their equivalents.

[0083] Of course, the present invention may have many other implementations. Based on this implementation, other implementations obtained by ordinary technicians in this field without any creative work are all within the scope of protection of the present invention.

Claims

1. A two-layer topological center clustering method, characterized in that: include: Acquire an original image, and perform data enhancement on the original image to obtain a first view and a second view; Inputting the first view and the second view into a classification model for processing respectively, and outputting multi-scale features; wherein the classification model is composed of an encoder, a low-level feature slot module, a high-level feature slot module, a reconstruction module, and a contrastive learning module; The multi-scale features are clustered to obtain a clustering result; based on the acquired real label set, a best matching result between the clustering result and the real label is generated.

2. A double-layer topological center clustering method according to claim 1, characterized in that: The first view and the second view are respectively input into a classification model for processing, and multi-scale features are output; wherein the classification model is composed of an encoder, a low-level feature slot module, a high-level feature slot module, a reconstruction module, and a contrastive learning module, specifically including the following steps: In the encoder, the first view and the second view are respectively input into the encoder, and the local feature h of the first view is output. 1 , global feature g 1 , and the local feature h of the second view 2 , global feature g 2 ; The local feature h output by the encoder 1 , the local feature h 2 Input low-level feature slot module S respectively l , output low-level features and The local feature h output by the encoder 1 , the local feature h 2 Input high-level feature slot module S respectively h , output high-level features and The reconstruction module also includes a decoder d, which converts the low-level feature output of the low-level feature slot module into and Input the decoder and reconstruct it into local feature h l1 and h l2 ; The reconstruction module optimizes the low-level feature slot module by reconstructing the loss function; Based on the global feature g 1 , global feature g 2 , low-level features Low-level features High-level features High-level features Feature pooling and fusion are performed to generate the multi-scale features.

3. A double-layer topological center clustering method according to claim 2, characterized in that: The contrastive learning module includes a contrastive learning loss function, and the contrastive learning loss function is used to perform view alignment; the contrastive learning loss function L SC It is expressed as: Where τ is the temperature coefficient; K is the number of high-level feature slots; k is the current slot index, which is used to traverse K slots when averaging the loss; k′ is the traversal index when summing the denominator, which is used to convert All slots in the same view Perform similarity comparison and obtain a softmax distribution; The high-level features of the first view indexed by the current slot; High-level features of the second view indexed by the current slot.

4. A double-layer topological center clustering method according to claim 1, characterized in that: The classification model also includes: using image scale contrast learning loss function L IC , unsupervised contrastive learning And supervised contrastive learning The multi-scale features are optimized; wherein the formula is expressed as: in, is the global feature of the first view of the i-th sample; is the global feature of the second view of the i-th sample; g n is the feature representation of the nth sample in the first view or the second view. When summing ∑n in the denominator, n traverses all samples except i; τ is the temperature coefficient; 1 [n≠i] is the indicator function, which is 1 when n≠i and 0 otherwise; is the index set of other samples with the same label as i. When the sample has a label, Select positive samples with the same label as i; The number of elements in is the number of other samples with the same label as i; B is the set of all sample indexes; is a labeled subset; λ is a weight coefficient used to control the unsupervised contrastive learning term and supervised contrastive learning the relative importance of .

5. A double-layer topological center clustering method according to claim 1, characterized in that: In the step of clustering the multi-scale features to obtain a clustering result; and generating the best matching result between the clustering result and the real label based on the acquired real label set, the following steps are specifically included: Clustering the multi-scale features by k-means; Divide all samples into several clusters; According to the Hungarian algorithm, the best matching result is selected between the cluster index set and the real label set, specifically: If the number of clusters is greater than the number of true labels, the extra clusters are assigned to the empty set, and all samples in the cluster are defined as prediction errors; If the number of clusters is less than the number of true labels, the excess labels are assigned to the empty set, and all samples corresponding to the label are defined as prediction errors.

6. A system according to the double-layer topological center clustering method of claim 1, characterized in that: include: An acquisition module: acquires an original image, performs data enhancement on the original image, and obtains a first view and a second view; Processing module: input the first view and the second view into a classification model for processing, and output multi-scale features; wherein the classification model is composed of an encoder, a low-level feature slot module, a high-level feature slot module, a reconstruction module, and a contrastive learning module; Clustering module: clustering the multi-scale features to obtain clustering results; based on the acquired real label set, generating the best matching result between the clustering results and the real labels.

7. A device, characterized in that: The device includes a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing a two-layer topological center clustering method as described in any one of claims 1-5; and the processor is used to execute the program instructions stored in the memory to implement a two-layer topological center clustering.

8. A storage medium, characterized in that: The device stores program instructions executable by a processor, wherein the program instructions are used to execute a double-layer topology center clustering method as described in any one of claims 1 to 5.

Images