A dictionary learning method and device for zero-sample recognition

Abstract

Disclosed is a dictionary learning method and device for zero-shot recognition, which can establish alignment between visual space and semantic space at the category level and image level, respectively, so as to achieve high-precision zero-shot image recognition. The method comprises: (1) training a cross-domain dictionary at the category level based on a cross-domain dictionary learning method; (2) generating semantic attributes of an image based on the category-level cross-domain dictionary learned in step (1); (3) training a cross-domain dictionary at the image level based on the image semantic attributes generated in step (2); (4) completing the recognition task of an image at an invisible category based on the image-level cross-domain dictionary learned in step (3).

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a dictionary learning method for zero-sample recognition and a dictionary learning device for zero-sample recognition. Background Art

[0002] Zero-Shot Learning (ZSL) originated from the problem of how to identify the category of an image when there is only a small amount of labeled data or no labeled data for certain categories, which was proposed by Larochelle et al. in 2008. Unlike traditional known category image recognition, which requires a large amount of data for training, zero-shot learning uses auxiliary knowledge to recognize images of unknown categories, which makes it possible to realize many practical application scenarios and also promotes the development of intelligent systems.

[0003] It is worth noting that the task of zero-shot learning is to correctly identify images of categories that have not appeared in the training set. For the above zero-shot learning task, Lampert et al. proposed the Animals with Attributes (AWA) dataset. The training set consists of images of visible categories and semantic attributes of all categories (including visible and invisible categories); the test set contains images of invisible categories.

[0004] From the perspective of achieving zero-shot recognition tasks, existing methods can be roughly divided into two categories: one is zero-shot learning based on unified spatial expression, and the other is zero-shot learning based on generative models.

[0005] Methods based on unified spatial expression usually project visual features and semantic attribute features into a unified embedding space, and use some distance measurement methods to classify the unseen category images projected into the embedding space. A representative work is the structured embedding method based on category attribute generation proposed by Jiang et al. in 2018. This method first generates category-level features in the visual space for the visible class, and then uses the category-level features in the visual space and the corresponding semantic attribute features of the category as input to train the cross-domain dictionary, that is, the cross-domain dictionary is used to establish the association between the visual space and the semantic space at the category level. For the unseen category images in the test set, this method proposes three classification methods: the first is to map the unseen category images and category semantic attributes to the embedding space for distance measurement; the second is to map the image to the embedding space and then generate the corresponding image-level semantic attributes through the semantic space dictionary, and then perform distance measurement with the category semantic attributes in the semantic space; the third is to map the category semantic attributes to the embedding space, and then generate category image features through the visual space dictionary, and then perform distance measurement with the unseen category images in the visual space.

[0006] Zero-shot learning methods based on generative models usually generate a large number of unseen category images and use the generated unseen category images to train classification models. With the rise of Generative Adversarial Nets (GAN), many researchers have taken advantage of its ability to generate samples to achieve zero-shot recognition tasks. For example, the method proposed by Xian et al. in 2018 borrowed the idea of ​​conditional GAN ​​to construct a generator and used category semantic attributes as conditions to generate visual features with strong discriminativeness. Ji et al. proposed a method for generating pseudo samples of unseen categories based on dictionary learning in 2019. This method first uses semantic attribute features to extract the relationship between visible and unseen categories, and then uses this relationship as a coefficient to weight the combination of visible category images to generate pseudo samples of unseen categories, and then uses pseudo samples to train unseen category classifiers.

[0007] Due to limited manpower constraints, existing datasets applied to zero-shot recognition tasks only provide category semantic attributes, but lack image semantic attributes. This results in some methods that seek the connection between visual space and semantic space only staying at the category level, thus limiting the accuracy of zero-shot recognition tasks. For example, the method proposed by Jiang et al. uses a cross-domain dictionary learning model to establish the connection between the visual space and the semantic space established at the category level. However, compared to images, the information extracted by the model learned at the category level is relatively simple. Only building a model at the category level will ignore fine-grained image information. Summary of the invention

[0008] In order to overcome the defects of the prior art, the technical problem to be solved by the present invention is to provide a dictionary learning method for zero-shot recognition, which can establish the alignment between the visual space and the semantic space at the category level and the image level respectively, thereby achieving high-precision zero-shot image recognition.

[0009] The technical solution of the present invention is: this dictionary learning method for zero-sample recognition comprises the following steps:

[0010] (1) Train the cross-domain dictionary at the category level based on the cross-domain dictionary learning method;

[0011] (2) Generate semantic attributes of the image based on the category-level cross-domain dictionary learned in step (1);

[0012] (3) training a cross-domain dictionary at the image level based on the image semantic attributes generated in step (2);

[0013] (4) Based on the image-level cross-domain dictionary learned in step (3), the recognition task of unseen category images is completed.

[0014] The present invention is based on a cross-domain dictionary learning model. By constraining the expressions of the sparse spaces projected to the visual space data and the semantic space data through the spatial dictionary to be consistent, the association between the visual space and the semantic space is established from the category layer and the image layer respectively. By adding a cross-domain dictionary at the image level, the alignment between the visual space and the semantic space can be established from the category layer and the image layer respectively, and image information that is more fine-grained than the category layer can be extracted, thereby achieving high-precision zero-sample image recognition.

[0015] A dictionary learning device for zero-sample recognition is also provided, the device comprising:

[0016] A first training module configured to train a cross-domain dictionary of a category layer based on a cross-domain dictionary learning method;

[0017] a generation module configured to generate semantic attributes of an image based on the category-level cross-domain dictionary learned by the first training module;

[0018] a second training module configured to train a cross-domain dictionary at the image level;

[0019] The recognition module is configured to complete the recognition task of the unseen category image based on the image-level cross-domain dictionary learned by the second training module. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a framework diagram of the present invention, where the wavy rectangular box is the data provided in the data set, and the rectangular box is the generated data. The figure shows three model training steps and one test step, where the test step shows the comparison in the semantic space.

[0021] Figure 2 A flow chart of a dictionary learning method for zero-sample recognition according to the present invention is shown. DETAILED DESCRIPTION

[0022] like Figure 2 As shown, this dictionary learning method for zero-sample recognition includes the following steps:

[0023] (1) Train the cross-domain dictionary at the category level based on the cross-domain dictionary learning method;

[0024] (2) Generate semantic attributes of the image based on the category-level cross-domain dictionary learned in step (1);

[0025] (3) training a cross-domain dictionary at the image level based on the image semantic attributes generated in step (2);

[0026] (4) Based on the image-level cross-domain dictionary learned in step (3), the recognition task of unseen category images is completed.

[0027] The present invention is based on a cross-domain dictionary learning model. By constraining the expressions of the sparse spaces projected to the visual space data and the semantic space data through the spatial dictionary to be consistent, the association between the visual space and the semantic space is established from the category layer and the image layer respectively. By adding a cross-domain dictionary at the image level, the alignment between the visual space and the semantic space can be established from the category layer and the image layer respectively, and image information that is more fine-grained than the category layer can be extracted, thereby achieving high-precision zero-sample image recognition.

[0028] Preferably, the step (1) comprises:

[0029] (1.1) Extract the category prototype P in the visual space by calculating the category center of the visible category image v , which is formula (1):

[0030]

[0031] Among them, Y v is the sample feature matrix; H is the sample label matrix;

[0032] (1.2) The category prototype P v and the category semantic attribute P s A pair of inputs is formed to train the cross-domain dictionary of the category layer. The connection between the visual space and the semantic space is established at the category level by constraining the category prototype and the category semantic attribute to share sparse coefficients. The specific expression is formula (2):

[0033]

[0034] Among them, the first term is the visual space dictionary reconstruction error term; the second term is the semantic space dictionary reconstruction error term; D v is the visual-spatial dictionary; D s is the semantic space dictionary; X p is the sparse coefficient matrix; λ is the harmonic parameter;

[0035] (1.3) In order to reduce the impact of the domain difference between visible and invisible categories on the model accuracy and improve the model's recognition ability for invisible category samples, an adaptive loss function for invisible categories is introduced, which is formula (3):

[0036]

[0037] in, is the category prototype of the unseen class to be solved; is the semantic attribute matrix of the unseen categories; is the sparse coefficient matrix corresponding to the unseen category;

[0038] The joint loss function of the category layer is formula (4):

[0039]

[0040] The training goal of the category layer is to minimize the loss function shown in formula (4). The variables to be solved include: visual space dictionary D v ; Semantic space dictionary D s ; Visible category prototype P v ; Invisible class prototype Visible class sparse coefficient x p ; Sparse coefficient of invisible class

[0041] Preferably, the step (2) comprises:

[0042] (2.1) Using the visual-spatial dictionary D υ Generate the sparse coefficients x of the image y , the specific expression is formula (5):

[0043]

[0044] Among them, the first term is the reconstruction error term; the second term is the constraint term, which constrains the generated image sparse coefficients and their categories to be based on the same visual space dictionary D υ The generated sparse coefficients are similar; w x is the harmonization parameter;

[0045] (2.2) Using the semantic space dictionary D s and the semantic attribute P of its category s Jointly generate image semantic attributes Y s , the specific expression is formula (6):

[0046]

[0047] Among them, w p is the reconciliation parameter.

[0048] Preferably, the step (3) comprises:

[0049] In order to further mine the information of the image and improve the generalization performance of the model, the image semantic attributes generated in step (2) are used to train the cross-domain dictionary of the image layer. The specific expression is formula (7):

[0050]

[0051] Among them, the first term is the reconstruction error term of the visual space; the second term is the reconstruction error term of the semantic space; and are the dictionaries of the visual space and semantic space at the image level respectively; X is the sparse coefficient; μ is the harmonization parameter.

[0052] Preferably, the step (4) comprises:

[0053] In terms of visual-spatial comparison:

[0054] Unseen Category Semantic Attributes First, through the image layer semantic space dictionary Generate sparse coefficients X u , which is formula (8):

[0055]

[0056] Reusing the Visual-Spatial Dictionary Generate a representation of the category in visual space Measure the test image and each category description separately The distance is used to determine the category of the test image according to the distance, which is formula (9):

[0057]

[0058] In terms of sparse domain comparison: the test image is expressed in sparse space according to the visual space dictionary of the image layer, which is formula (10):

[0059]

[0060] Metric x u and the description of each category in sparse space X u The distance between [c] and the category closest to the test image is the category of the image, which is formula (11):

[0061]

[0062] In terms of semantic space comparison:

[0063] First, the test image is encoded according to the visual space dictionary of the image layer to obtain x u ; Then generate the semantic attributes of the image based on the semantic space dictionary of the image layer Metric y s The distance from the semantic attributes of each category and the category of the test image are determined based on the distance, which is formula (12):

[0064]

[0065] Preferably, the method is experimented on two image datasets based on zero-shot recognition tasks: the AwA dataset and the aPY dataset, and the recognition accuracy is compared with the current mainstream zero-shot recognition models, including SJE, EZSL, SYNC, SAE, CDL, ALE, CONSE, LATEM, and DEVISE; AwA is an animal image dataset containing 50 animal categories and 30,475 images, each class has 85 annotated attributes; the standard division of the zero-shot recognition experiment is to use 40 categories as visible categories and the other 10 categories as invisible categories.

[0066] A dictionary learning device for zero-sample recognition is also provided, the device comprising:

[0067] A first training module configured to train a cross-domain dictionary of a category layer based on a cross-domain dictionary learning method;

[0068] a generation module configured to generate semantic attributes of an image based on the category-level cross-domain dictionary learned by the first training module;

[0069] a second training module configured to train a cross-domain dictionary at the image level;

[0070] The recognition module is configured to complete the recognition task of the unseen category image based on the image-level cross-domain dictionary learned by the second training module.

[0071] Preferably, the first training module performs:

[0072] Extract the category prototype P in the visual space by calculating the category center of the visible category image v , which is formula (1):

[0073]

[0074] Among them, Y v is the sample feature matrix; H is the sample label matrix;

[0075] The category prototype P v and the category semantic attribute P s A pair of inputs is formed to train the cross-domain dictionary of the category layer. The connection between the visual space and the semantic space is established at the category level by constraining the category prototype and the category semantic attribute to share sparse coefficients. The specific expression is formula (2):

[0076]

[0077] Among them, the first term is the visual space dictionary reconstruction error term; the second term is the semantic space dictionary reconstruction error term; D v is the visual-spatial dictionary; D s is the semantic space dictionary; Xp is the sparse coefficient matrix; λ is the harmonic parameter;

[0078] In order to reduce the impact of the domain difference between visible and invisible categories on the model accuracy and improve the model's recognition ability for invisible category samples, an adaptive loss function for invisible categories is introduced, which is formula (3):

[0079]

[0080] in, is the category prototype of the unseen class to be solved; is the semantic attribute matrix of the unseen categories; is the sparse coefficient matrix corresponding to the unseen category;

[0081] The joint loss function of the category layer is formula (4):

[0082]

[0083] The training goal of the category layer is to minimize the loss function shown in formula (4). The variables to be solved include: visual space dictionary D v ; Semantic space dictionary D s ; Visible category prototype P v ; Invisible class prototype Visible class sparse coefficient X p ; Sparse coefficient of invisible class

[0084] Preferably, the generating module performs:

[0085] Using the visual spatial dictionary v Generate the sparse coefficients x of the image y , the specific expression is formula (5):

[0086]

[0087] Among them, the first term is the reconstruction error term; the second term is the constraint term, which constrains the generated image sparse coefficients and their categories to be based on the same visual space dictionary D v The generated sparse coefficients are similar; w x is the harmonization parameter;

[0088] Using the semantic space dictionary D s and the semantic attribute P of its category s Jointly generate image semantic attributes Y s , the specific expression is formula (6):

[0089]

[0090] Among them, w p is the harmonization parameter;

[0091] The second training module performs:

[0092] In order to further mine the information of the image and improve the generalization performance of the model, the image semantic attributes generated in step (2) are used to train the cross-domain dictionary of the image layer. The specific expression is formula (7):

[0093]

[0094] Among them, the first term is the reconstruction error term of the visual space; the second term is the reconstruction error term of the semantic space; and are the dictionaries of the visual space and semantic space at the image level respectively; X is the sparse coefficient; μ is the harmonization parameter.

[0095] Preferably, the identification module performs:

[0096] In terms of visual-spatial comparison:

[0097] Unseen Category Semantic Attributes First, through the image layer semantic space dictionary Generate sparse coefficients x u , which is formula (8):

[0098]

[0099] Reusing the Visual-Spatial Dictionary Generate a representation of the category in visual space Measure the test image and each category description separately The distance is used to determine the category of the test image according to the distance, which is formula (9):

[0100]

[0101] In terms of sparse domain comparison: the test image is expressed in sparse space according to the visual space dictionary of the image layer, which is formula (10):

[0102]

[0103] Metric x u and the description of each category in sparse space X u The distance between [c] and the category closest to the test image is the category of the image, which is formula (11):

[0104]

[0105] In terms of semantic space comparison:

[0106] First, the test image is encoded according to the visual space dictionary of the image layer to obtain x u ; Then generate the semantic attributes of the image based on the semantic space dictionary of the image layer Metric y s The distance from the semantic attributes of each category and the category of the test image are determined based on the distance, which is formula (12):

[0107]

[0108] To test the effectiveness of the proposed method, the present invention conducts experiments on two image datasets based on zero-shot recognition tasks (AwA dataset and aPY dataset), and compares the recognition accuracy with the current mainstream zero-shot recognition models, including SJE, EZSL, SYNC, SAE, CDL, ALE, CONSE, LATEM, and DEVISE. Tables 1 and 2 respectively give the comparison of the zero-shot recognition accuracy of the proposed method in the present invention and other existing methods on the two datasets.

[0109] AwA is an animal image dataset that contains 50 animal categories and 30,475 images. Each class has 85 annotated attributes. The standard split for zero-shot recognition experiments is to use 40 categories as seen categories and the other 10 categories as unseen categories.

[0110] The present invention uses resnet101 to extract image features, and the feature dimension is 2048. There are 40 atoms in the category layer cross-domain dictionary, 200 atoms in the image layer cross-domain dictionary, parameter λ=1, parameter α=1, parameter β=1, parameter μ=1, parameter w x =1, parameter w p =1e -10 The recognition accuracy of the method proposed by the present invention and the comparative method is shown in Table 1. It can be seen that on this data set, the method proposed by the present invention achieves the highest accuracy.

[0111] Table 1

[0112]

[0113] The aPY dataset contains 32 classes and 15,339 images, each of which has 64-dimensional semantic attributes. The present invention regards 20 classes as visible classes and the other 12 classes as invisible classes according to the standard classification method.

[0114] The present invention uses resnet101 to extract image features, and the feature dimension is 2048. There are 20 atoms in the category layer cross-domain dictionary, 200 atoms in the image layer cross-domain dictionary, parameter λ=1, parameter α=1, parameter β=1, parameter μ=1, parameter wx =1, parameter w p = 1. The recognition accuracy of the method proposed by the present invention and the comparative method is shown in Table 2. It can be seen that on this data set, the method proposed by the present invention achieves the highest accuracy.

[0115] Table 2

[0116]

[0117] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the protection scope of the technical solution of the present invention.

Claims

1. A dictionary learning method for zero-shot recognition, characterized by: The method comprises the following steps: (1) Train the cross-domain dictionary at the category level based on the cross-domain dictionary learning method; (2) Generate semantic attributes of the image based on the category-level cross-domain dictionary learned in step (1); (3) training a cross-domain dictionary at the image level based on the image semantic attributes generated in step (2); (4) Based on the image-level cross-domain dictionary learned in step (3), the recognition task of the unseen category images is completed; The step (1) comprises: (1.1) Extract the category prototype P in the visual space by calculating the category center of the visible category image v , which is formula (1): Among them, Y v is the sample feature matrix; H is the sample label matrix; (1.2) The category prototype P v and the category semantic attribute P s A pair of inputs is formed to train the cross-domain dictionary of the category layer. The connection between the visual space and the semantic space is established at the category level by constraining the category prototype and the category semantic attribute to share sparse coefficients. The specific expression is formula (2): Among them, the first term is the visual space dictionary reconstruction error term; the second term is the semantic space dictionary reconstruction error term; D v is the visual-spatial dictionary; D s is the semantic space dictionary; X p is the sparse coefficient matrix; λ is the harmonic parameter; (1.3) In order to reduce the impact of the domain difference between visible and invisible categories on the model accuracy and improve the model's recognition ability for invisible category samples, an adaptive loss function for invisible categories is introduced, which is formula (3): in, is the category prototype of the unseen class to be solved; To be invisible Semantic attribute matrix of categories; is the sparse coefficient corresponding to the unseen category matrix; The joint loss function of the category layer is formula (4): The training goal of the category layer is to minimize the loss function shown in formula (4). The variables to be solved include: visual space dictionary D v ; Semantic space dictionary D s ; Visible category prototype P v ; Invisible class prototype Visible class sparse coefficient X p ; Sparse coefficient of invisible class 2. The dictionary learning method for zero-sample recognition according to claim 1, characterized in that: The step (2) comprises: (2.1) Using the visual-spatial dictionary D v Generate the sparse coefficients X of the image y , the specific expression is formula (5): Among them, the first term is the reconstruction error term; the second term is the constraint term, which constrains the generated image sparse coefficients and their categories to be based on the same visual space dictionary D v The generated sparse coefficients are similar; w x is the harmonization parameter; (2.2) Using the semantic space dictionary D s and the semantic attribute P of its category s Jointly generate image semantic attributes Y s , the specific expression is formula (6): Among them, w p is the reconciliation parameter.

3. The dictionary learning method for zero-sample recognition according to claim 2, characterized in that: The step (3) comprises: In order to further mine the information of the image and improve the generalization performance of the model, the image semantic attributes generated in step (2) are used to train the cross-domain dictionary of the image layer. The specific expression is formula (7): Among them, the first term is the reconstruction error term of the visual space; the second term is the reconstruction error term of the semantic space; and are the dictionaries of the visual space and semantic space at the image level respectively; X is the sparse coefficient; μ is the harmonization parameter.

4. The dictionary learning method for zero-sample recognition according to claim 3, characterized in that: The step (4) comprises: In terms of visual-spatial comparison: Unseen Category Semantic Attributes First, through the image layer semantic space dictionary Generate sparse coefficients X u , which is formula (8): Reusing the Visual-Spatial Dictionary Generate a representation of the category in visual space Measure the test image and each category description separately The distance is used to determine the category of the test image according to the distance, which is formula (9): In terms of sparse domain comparison: the test image is expressed in sparse space according to the visual space dictionary of the image layer, which is formula (10): Metric x u and the description of each category in sparse space X u The distance between [c] and the category closest to the test image is the category of the image, which is formula (11): In terms of semantic space comparison: First, the test image is encoded according to the visual space dictionary of the image layer to obtain x u ; Then generate the semantic attributes of the image based on the semantic space dictionary of the image layer Metric y s The distance from the semantic attributes of each category and the category of the test image are determined based on the distance, which is formula (12):

5. The dictionary learning method for zero-sample recognition according to claim 4, characterized in that: This method is experimented on two image datasets based on zero-shot recognition tasks: the AwA dataset and the aPY dataset, and the recognition accuracy is compared with the current mainstream zero-shot recognition models, including SJE, EZSL, SYNC, SAE, CDL, ALE, CONSE, LATEM, and DEVISE; AwA is an animal image dataset that contains 50 animal categories and 30,475 images, each class has 85 annotated attributes; The standard split for zero-shot recognition experiments is to use 40 categories as seen categories and the other 10 categories as unseen categories.

6. A dictionary learning device for zero-sample recognition, characterized in that: The device includes: A first training module configured to train a cross-domain dictionary of a category layer based on a cross-domain dictionary learning method; a generation module configured to generate semantic attributes of an image based on the category-level cross-domain dictionary learned by the first training module; a second training module configured to train a cross-domain dictionary at the image level; A recognition module configured to complete a recognition task of an image of an unseen category based on the image-level cross-domain dictionary learned by the second training module; The first training module performs: Extract the category prototype P in the visual space by calculating the category center of the visible category image v , which is formula (1): Among them, Y v is the sample feature matrix; H is the sample label matrix; The category prototype P v and the category semantic attribute P s A pair of inputs is formed to train the cross-domain dictionary of the category layer. The connection between the visual space and the semantic space is established at the category level by constraining the category prototype and the category semantic attribute to share sparse coefficients. The specific expression is formula (2): Among them, the first term is the visual space dictionary reconstruction error term; the second term is the semantic space dictionary reconstruction error term; D v is the visual-spatial dictionary; D s is the semantic space dictionary; X p is the sparse coefficient matrix; λ is the harmonic parameter; In order to reduce the impact of the domain difference between visible and invisible categories on the model accuracy and improve the model's recognition ability for invisible category samples, an adaptive loss function for invisible categories is introduced, which is formula (3): in, is the category prototype of the unseen class to be solved; is the semantic attribute matrix of the unseen categories; is the sparse coefficient matrix corresponding to the unseen category; The joint loss function of the category layer is formula (4): The training goal of the category layer is to minimize the loss function shown in formula (4). The variables to be solved include: visual space dictionary D v ; Semantic space dictionary D s ; Visible category prototype P v ; Invisible class prototype Visible class sparse coefficient X p ; Sparse coefficient of invisible class 7. The dictionary learning device for zero-sample recognition according to claim 6, characterized in that: The generation module performs: Using the visual spatial dictionary v Generate the sparse coefficients X of the image y , the specific expression is formula (5): Among them, the first term is the reconstruction error term; the second term is the constraint term, which constrains the generated image sparse coefficients and their categories to be based on the same visual space dictionary D v The generated sparse coefficients are similar; w x is the harmonization parameter; Using the semantic space dictionary D s and the semantic attribute P of its category s Jointly generate image semantic attributes Y s , the specific expression is formula (6): Among them, w p is the harmonization parameter; The second training module performs: In order to further mine the information of the image and improve the generalization performance of the model, the image semantic attributes generated in step (2) are used to train the cross-domain dictionary of the image layer. The specific expression is formula (7): Among them, the first term is the reconstruction error term of the visual space; the second term is the reconstruction error term of the semantic space; and are the dictionaries of the visual space and semantic space at the image level respectively; X is the sparse coefficient; μ is the harmonization parameter.

8. The dictionary learning device for zero-sample recognition according to claim 7, characterized in that: The identification module performs: In terms of visual-spatial comparison: Unseen Category Semantic Attributes First, through the image layer semantic space dictionary Generate sparse coefficients X u , which is formula (8): Reusing the Visual-Spatial Dictionary Generate a representation of the category in visual space Measure the test image and each category description separately The distance is used to determine the category of the test image according to the distance, which is formula (9): In terms of sparse domain comparison: the test image is expressed in sparse space according to the visual space dictionary of the image layer, which is formula (10): Metric x u and the description of each category in sparse space X u The distance between [c] and the category closest to the test image is the category of the image, which is formula (11): In terms of semantic space comparison: First, the test image is encoded according to the visual space dictionary of the image layer to obtain x u ; Then generate the semantic attributes of the image based on the semantic space dictionary of the image layer Metric y s The distance from the semantic attributes of each category and the category of the test image are determined based on the distance, which is formula (12):

Images