A method, device, equipment and storage medium for generating multi-classification model of image

Abstract

The present application discloses a method, apparatus, device and storage medium for generating an image multi-classification model, and relates to the field of computer vision technology. First, category labels are assigned to images in a public data set to form an image set, and then the images in the image set are divided into sub-blocks and thumbnails, and are respectively sent to an encoder for processing after feature extraction, and a hierarchical cross-modal attention module is constructed. Then, the training image is input into the hierarchical cross-modal attention module and then spliced ​​with the encoding prompt, and the feature extraction network model and the decoder are input to obtain the decoded output. Finally, the cross entropy loss of the decoded output is calculated and weighted summed, and the model parameters are optimized by gradient descent to obtain an image multi-classification model. The image multi-classification model can be used to achieve high-precision image multi-classification.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and more specifically, to a method, apparatus, device and storage medium for generating a multi-classification model of an image. Background Art

[0002] Image classification is a basic and important task in the field of computer vision, which aims to enable computers to understand the information of input images and classify them. Traditional image classification tasks usually only involve the prediction of a single category label, while image multi-classification tasks can obtain classification results of multiple dimensions of an image with a single input. This multi-classification task has a wide range of needs in practical applications. For example, in the fields of medical image analysis, autonomous driving, and product identification, images need to be classified and identified in many aspects.

[0003] Although the importance of image multi-classification tasks has become increasingly prominent, and many researchers have made significant contributions in this field, the existing multi-classification algorithms still have some shortcomings. The main problem is that these algorithms do not process image features deeply enough, resulting in unsatisfactory results for multi-classification tasks. Specifically: 1. Insufficient feature extraction: Existing multi-classification algorithms often rely on a single feature extraction method, which cannot fully capture the multi-scale and multi-level information in the image, thus affecting the accuracy and robustness of classification. 2. Insufficient use of cross-modal information: Image data usually contains multi-modal information (such as color, texture, shape, etc.), but most algorithms fail to effectively integrate these cross-modal information, resulting in limited classification results. 3. Lack of attention mechanism: When dealing with complex multi-classification tasks, there is a lack of effective attention mechanism to highlight key areas and features, resulting in poor performance of the model when facing complex scenes. 4. Low model training efficiency: During the training process, the existing multi-classification algorithms have long training time and slow convergence speed due to insufficient feature representation and limitations of optimization methods, which makes it difficult to meet the high-efficiency requirements in practical applications.

[0004] Therefore, most of the current classification algorithms do not process image features deeply enough, resulting in unsatisfactory results in multi-classification tasks. Summary of the invention

[0005] The purpose of this application is to provide a method, device, equipment and storage medium for generating an image multi-classification model in order to overcome the existing technical defects. The image multi-classification model can be used to achieve high-precision image multi-classification.

[0006] The purpose of this application is achieved through the following technical solutions:

[0007] In a first aspect, the present application proposes a method for generating a multi-classification model of an image, the method comprising:

[0008] Assign category labels to images in public datasets to form image collections;

[0009] The images in the image collection are divided into sub-blocks and thumbnails, and are sent to the encoder for processing after feature extraction to build a hierarchical cross-modal attention module;

[0010] The training image is input into the hierarchical cross-modal attention module and then concatenated with the encoding hint, and then input into the feature extraction network model and decoder to obtain the decoded output;

[0011] The cross entropy loss of the decoded output is calculated and weighted summed, and the model parameters are optimized by gradient descent to obtain the image multi-classification model.

[0012] In a possible implementation, the images in the image set are divided into sub-blocks and thumbnails, and are respectively sent to the encoder for processing after feature extraction, and the step of constructing the hierarchical cross-modal attention module includes:

[0013] Dividing an image in the image set into a plurality of sub-blocks;

[0014] Calculate the images in the image set to obtain thumbnails;

[0015] The sub-block and thumbnail are respectively input into the feature extraction network model to obtain the sub-block features and thumbnail features;

[0016] Use an encoder to process the sub-block features and the thumbnail features to obtain sub-block encoding features and thumbnail encoding features respectively;

[0017] Adding a sub-block identifier before and after the sub-block coding feature to obtain a sub-block unique coding feature, and adding a thumbnail identifier before and after the thumbnail coding feature to obtain a thumbnail unique coding feature;

[0018] The hierarchical cross-modal attention features are composed according to the unique coding features of the sub-blocks and the unique coding features of the thumbnails, and the construction of the hierarchical cross-modal attention module is completed.

[0019] In one possible implementation, the thumbnail The calculation method is ,in is the width of the thumbnail, , is the height of the thumbnail, , is the width of the sub-block, is the scale resolution, is the block size, For rounding operation, To obtain the maximum value, To image Resize the image to the height and width of the thumbnail.

[0020] In a possible implementation, the step of inputting the training image into the hierarchical cross-modal attention module, concatenating the image with the encoding hint, and inputting the image into the feature extraction network model and the decoder to obtain the decoded output includes:

[0021] Input the training image into the hierarchical cross-modal attention module to obtain the hierarchical cross-modal attention features of the training image;

[0022] The hierarchical cross-modal attention features of the training images and the encoded prompt set are concatenated to obtain concatenated features;

[0023] The concatenated features are input into the feature extraction network model and then sent to the decoder to obtain the decoded output.

[0024] In a possible implementation, before the step of inputting the training image into the hierarchical cross-modal attention module to obtain the hierarchical cross-modal attention features of the training image, the step further includes:

[0025] A prompt set is constructed according to the number of multi-classification tasks, and then the prompts in the prompt set are encoded by an encoder to obtain an encoded prompt set.

[0026] In a possible implementation, the steps of calculating the cross entropy loss of the decoded output and weighted summing the results, and optimizing the model parameters by gradient descent to obtain the image multi-classification model include:

[0027] Input the decoded output and the labels corresponding to the multi-classification task into the cross entropy function to obtain the loss function;

[0028] Calculate the loss function of all categories and perform weighted summation to obtain the model loss score;

[0029] During the back propagation of model parameters, the model parameters are optimized by gradient descent, so that the model loss score is reduced until the model training is completed and an image multi-classification model is obtained.

[0030] In a second aspect, the present application proposes a device for generating a multi-classification model of an image, the device comprising:

[0031] An assignment module is used to assign category labels to images in a public dataset to form an image collection;

[0032] The feature extraction module is used to divide the images in the image set into sub-blocks and thumbnails, and send them to the encoder for processing after feature extraction to build a hierarchical cross-modal attention module;

[0033] The decoding output module is used to input the training image into the hierarchical cross-modal attention module and then concatenate it with the encoding prompt, and input it into the feature extraction network model and the decoder to obtain the decoding output;

[0034] The weighted summation module is used to calculate the cross entropy loss of the decoded output and perform weighted summation, and optimize the model parameters through gradient descent to obtain the image multi-classification model.

[0035] In a possible implementation, the feature extraction module is further used to:

[0036] Dividing an image in the image set into a plurality of sub-blocks;

[0037] Calculate the images in the image set to obtain thumbnails;

[0038] The sub-block and thumbnail are respectively input into the feature extraction network model to obtain the sub-block features and thumbnail features;

[0039] Use an encoder to process the sub-block features and the thumbnail features to obtain sub-block encoding features and thumbnail encoding features respectively;

[0040] Adding a sub-block identifier before and after the sub-block coding feature to obtain a sub-block unique coding feature, and adding a thumbnail identifier before and after the thumbnail coding feature to obtain a thumbnail unique coding feature;

[0041] The hierarchical cross-modal attention features are composed according to the unique coding features of the sub-blocks and the unique coding features of the thumbnails, and the construction of the hierarchical cross-modal attention module is completed.

[0042] In a third aspect, the present application also proposes a computer device, comprising a processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the image multi-classification model generation method as described in any one of the first aspects.

[0043] In a fourth aspect, the present application further proposes a computer-readable storage medium, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the image multi-classification model generation method as described in any one of the first aspects.

[0044] The above main scheme of the present application and its further options can be freely combined to form multiple schemes, all of which are schemes that can be adopted and claimed for protection in the present application; and in the present application, (non-conflicting options) options and other options can also be freely combined. After understanding the scheme of the present application, those skilled in the art can understand that there are many combinations based on the existing technology and common knowledge, all of which are technical schemes to be protected by the present application, and they are not exhaustively listed here.

[0045] The present application discloses a method, apparatus, device and storage medium for generating an image multi-classification model, and relates to the field of computer vision technology. First, category labels are assigned to images in a public data set to form an image set, and then the images in the image set are divided into sub-blocks and thumbnails, and are respectively sent to an encoder for processing after feature extraction, and a hierarchical cross-modal attention module is constructed. Then, the training image is input into the hierarchical cross-modal attention module and then spliced ​​with the encoding prompt, and the feature extraction network model and the decoder are input to obtain the decoded output. Finally, the cross entropy loss of the decoded output is calculated and weighted summed, and the model parameters are optimized by gradient descent to obtain an image multi-classification model. The image multi-classification model can be used to achieve high-precision image multi-classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A flow chart of a method for generating a multi-classification model of an image proposed in an embodiment of the present application is shown.

[0048] Figure 2 A construction diagram of the hierarchical cross-modal attention module proposed in an embodiment of the present application is shown.

[0049] Figure 3 A flowchart of cross-modal image multi-classification training proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0050] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0051] Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of this application.

[0052] In the prior art, most current classification algorithms do not process image features deeply enough, resulting in unsatisfactory results for multi-classification tasks.

[0053] Therefore, in order to solve the above problems, the embodiments of the present application provide an image multi-classification model generation method, device, equipment and storage medium. First, an image data set for image multi-classification is constructed, and then a hierarchical attention mechanism module is constructed to obtain rich image features. Finally, the image features obtained by the hierarchical attention mechanism module are used to implement the training of the multi-classification image classification model. The image multi-classification model can be obtained, and high-precision image multi-classification can be achieved to ensure the model's accurate image multi-classification capability.

[0054] Please refer to Figure 1 , Figure 1 A schematic diagram of a process for generating a multi-classification model of an image proposed in an embodiment of the present application is shown, and the method includes the following steps:

[0055] Step S1: Assign category labels to images in the public dataset to form an image set.

[0056] First, we need to collect images from public datasets, which will form the image set DATA. Then, for each image in the image set DATA , is the number of images in the image set DATA, ,in Represents the total number of images and is a positive integer, assigning a set of category labels to each image , Represents the number of categories in the multi-classification task, which is also a positive integer. This ensures that each image is correctly labeled with the corresponding category label, thus forming a set of images with category labels.

[0057] Step S2: Divide the images in the image set into sub-blocks and thumbnails, and send them to the encoder for processing after feature extraction to construct a hierarchical cross-modal attention module.

[0058] Each image in the image collection is divided into multiple sub-blocks, and a thumbnail of each image is generated. Then, the sub-blocks and thumbnails are respectively subjected to feature extraction, wherein the feature extraction uses a feature extraction network model, which adopts a deep learning model (Vision Transformer, ViT) based on the Transformer architecture. The deep learning model divides the image into fixed-size blocks and linearly embeds these blocks into a sequence, so that the self-attention mechanism of the Transformer can be used to process the image data.

[0059] The extracted features are then sent to the encoder for further processing to generate encoded features. Finally, these encoded features are combined to construct a hierarchical cross-modal attention module to enhance the model's attention to different parts and overall features and improve the model's performance.

[0060] Step S2 includes:

[0061] Step S21, dividing the image in the image set into a plurality of sub-blocks;

[0062] Step S22, calculating the images in the image set to obtain thumbnails;

[0063] Step S23, respectively inputting the sub-block and the thumbnail into the feature extraction network model to obtain the sub-block feature and the thumbnail feature;

[0064] Step S24: using an encoder to process the sub-block features and the thumbnail features to obtain sub-block coding features and thumbnail coding features respectively;

[0065] Step S25, respectively adding a sub-block identifier before and after the sub-block coding feature to obtain a sub-block unique coding feature, and adding a thumbnail identifier before and after the thumbnail coding feature to obtain a thumbnail unique coding feature;

[0066] Step S26: A hierarchical cross-modal attention feature is formed according to the unique coding features of the sub-blocks and the unique coding features of the thumbnails, thereby completing the construction of the hierarchical cross-modal attention module.

[0067] Get each image in the image collection DATA Width and height ,Will Divide into 8 sub-blocks evenly , where the width of each sub-block is ,high , and then calculate the image Thumbnail of , and the thumbnail And each sub-block Input them into the feature extraction network model respectively to obtain thumbnail features and sub-block features , Indicates that the feature extraction network model is used to perform feature extraction operations and the encoder T is used to process thumbnail features and sub-block features Generate sub-block encoding features and thumbnail encoding features , Indicates that the encoder T is used to process the features.

[0068] Thumbnail features after encoding Add special identifiers before and after Get unique coding feature , after each encoded sub-block feature Add special identifiers before and after to obtain the unique coding characteristics of each sub-block , uniquely encode the thumbnail features and unique encoding features of all sub-blocks Add together to get the hierarchical cross-modal attention features , complete the construction of the Graded Cross-Modal Attention Module (GA), which is a technology used to improve the processing effect of multimodal data. In computer vision tasks, especially in scenarios involving multimodal data such as images and text, it can help the model better capture and integrate information between different modalities. The graded cross-modal attention feature is a comprehensive feature vector that combines the global features (thumbnail features) and local features (sub-block features) of the image, which helps the model better capture the local and global features of the image and improve the performance of the model.

[0069] Figure 2 The construction diagram of the hierarchical cross-modal attention module proposed in the embodiment of the present application is shown, and the input image (each image in the image set) ) is divided into multiple sub-images, and then each sub-image sub_patch_1, sub_patch_2, ..., sub_patch_cls and thumbnail is sent to the feature extraction network model and encoder for further processing, and a special identifier is added before and after the encoded thumbnail features. Get unique coding feature , add the corresponding identifier patch_1, patch_2, ..., patch_cls before and after each sub-image sub_patch_1, sub_patch_2, ..., sub_patch_cls to obtain the corresponding encoding features otf_sub_patch_1, otf_sub_patch_2, ..., otf_sub_patch_sp, and finally add all the features to get the final hierarchical cross-modal attention feature.

[0070] Thumbnail The calculation method is ,in is the width of the thumbnail, , is the height of the thumbnail, , is the width of the sub-block, is the scale resolution, is the block size, For rounding operation, To obtain the maximum value, To image Resize the image to the height and width of the thumbnail.

[0071] Step S3: input the training image into the hierarchical cross-modal attention module and then concatenate it with the encoding prompt, input it into the feature extraction network model and decoder to obtain the decoded output.

[0072] The training image is input into the constructed hierarchical cross-modal attention module, which processes the input training image to generate hierarchical cross-modal attention features for the training image. The generated hierarchical cross-modal attention features for the training image are concatenated with encoding prompts (e.g., pre-defined text or label information) to combine the image features with the additional information. The concatenated features are then input into the feature extraction network model for further feature extraction. The extracted features are then sent to the decoder, which generates the final decoding output based on these features.

[0073] Step S3 includes:

[0074] Step S31, inputting the training image into the hierarchical cross-modal attention module to obtain the hierarchical cross-modal attention features of the training image;

[0075] Step S32, concatenating the hierarchical cross-modal attention features of the training image and the encoded prompt set to obtain a concatenated feature;

[0076] Step S33: input the concatenated features into the feature extraction network model and then send them into the decoder to obtain a decoded output.

[0077] For the input training image , through the hierarchical cross-modal attention module, the hierarchical cross-modal attention features of the training image are obtained , for each category k in the multi-classification task, the hierarchical cross-modal attention features With the encoded prompt Perform splicing to obtain splicing features ,in Indicates the splicing operation, which splices the features Input into the feature extraction network model to extract features, and then send the extracted features to the decoder D for decoding to obtain the final decoded output , Indicates decoding by decoder D.

[0078] Figure 3The flowchart of the cross-modal image multi-classification training proposed in the embodiment of the present application is shown. The input training image passes through a hierarchical cross-modal attention module, and the output is a hierarchical cross-modal attention feature. The hierarchical cross-modal attention feature is divided into multiple parts, which are then sent to different branches for processing and compared with the encoding prompt t_pmt k Perform splicing to obtain the splicing feature S k , extract the features and send them to the decoder D for decoding to get the output O_S k , will output O_S k The label corresponding to this task is O_S cls Input to the cross entropy function to get the loss function Score cls , then the labels are further converted into scores, all scores are aggregated into a loss function, and the total loss value Loss is calculated. According to the loss value, the parameters of the model will be updated to optimize the model performance, and finally a complete multi-classification model is formed.

[0079] Before step S31, the method further includes:

[0080] A prompt set is constructed according to the number of multi-classification tasks, and then the prompts in the prompt set are encoded by an encoder to obtain an encoded prompt set.

[0081] According to the number of classification tasks (number of categories cls), construct a prompt set , where cls is a positive integer, each prompt pmtk corresponds to a category, and the encoder T is used to encode each prompt in the prompt set P to obtain the encoded prompt set .

[0082] Step S4: Calculate the cross entropy loss of the decoded output and perform weighted summation, and optimize the model parameters by gradient descent to obtain the image multi-classification model.

[0083] The cross entropy loss function is used to measure the difference between the model prediction and the true label. The losses of different categories are weighted to deal with the problem of category imbalance. The gradient descent algorithm is used to update the model parameters and minimize the loss function. Finally, through multiple iterative optimizations, a good image multi-classification model is finally obtained.

[0084] Step S4 includes:

[0085] Step S41, input the decoded output and the label corresponding to the multi-classification task into the cross entropy function to obtain the loss function;

[0086] Step S42: Calculate the loss functions of all categories and perform weighted summation to obtain the model loss score;

[0087] Step S43: Optimize the model parameters by gradient descent during back propagation of the model parameters, so that the model loss score is reduced until the model training is completed to obtain an image multi-classification model.

[0088] After getting the decoded output for each category After that, there is a true label for each category , which is a one-hot encoded vector representing the actual category, and the decoded output and the true label Input into the cross entropy function to calculate the loss function for each category ,in is the cross entropy loss function, and then the loss function of all categories Perform weighted summation to get the total loss score of the model ,in It is The weighted score of each category is a positive number and is used to deal with the problem of category imbalance.

[0089] The model parameters are optimized by gradient descent. The gradient of the loss function with respect to the model parameters is calculated, and the parameters are updated in the opposite direction of the gradient to gradually reduce the value of the loss function. Through multiple iterations, the model parameters are gradually optimized to achieve the best performance on the training data. When the change in the loss function is less than a certain threshold or reaches a predetermined number of iterations, the model is considered to have converged and the training process is completed.

[0090] A possible implementation of an image multi-classification model generation device is provided below, which is used to execute the various execution steps and corresponding technical effects of the image multi-classification model generation method shown in the above embodiment and possible implementation. The device includes:

[0091] An assignment module is used to assign category labels to images in a public dataset to form an image collection;

[0092] The feature extraction module is used to divide the images in the image set into sub-blocks and thumbnails, and send them to the encoder for processing after feature extraction to build a hierarchical cross-modal attention module;

[0093] The decoding output module is used to input the training image into the hierarchical cross-modal attention module and then concatenate it with the encoding prompt, and input it into the feature extraction network model and the decoder to obtain the decoding output;

[0094] The weighted summation module is used to calculate the cross entropy loss of the decoded output and perform weighted summation, and optimize the model parameters through gradient descent to obtain the image multi-classification model.

[0095] In a possible implementation, the feature extraction module is further used to:

[0096] Dividing an image in the image set into a plurality of sub-blocks;

[0097] Calculate the images in the image set to obtain thumbnails;

[0098] The sub-block and thumbnail are respectively input into the feature extraction network model to obtain the sub-block features and thumbnail features;

[0099] Use the encoder to process the sub-block features and the thumbnail features to obtain the sub-block encoding features and the thumbnail encoding features respectively;

[0100] Adding a sub-block identifier before and after the sub-block coding feature to obtain a sub-block unique coding feature, and adding a thumbnail identifier before and after the thumbnail coding feature to obtain a thumbnail unique coding feature;

[0101] The hierarchical cross-modal attention features are composed according to the unique coding features of the sub-blocks and the unique coding features of the thumbnails, and the construction of the hierarchical cross-modal attention module is completed.

[0102] This preferred embodiment provides a computer device, which can implement the steps in any embodiment of the image multi-classification model generation method provided in the embodiments of the present application. Therefore, the beneficial effects of the image multi-classification model generation method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0103] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling related hardware through instructions, and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present application provides a storage medium in which multiple instructions are stored, and the instructions can be loaded by a processor to execute the steps of any embodiment of the method for generating a multi-classification model of an image provided in the embodiment of the present application.

[0104] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0105] Since the instructions stored in the storage medium can execute the steps in any image multi-classification model generation method embodiment provided in the embodiments of the present application, the beneficial effects that can be achieved by any image multi-classification model generation method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0106] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for generating a multi-classification model of an image, characterized in that: The method comprises: Assign category labels to images in public datasets to form image collections; Dividing an image in the image set into a plurality of sub-blocks; Calculate the images in the image set to obtain thumbnails; The sub-block and thumbnail are respectively input into the feature extraction network model to obtain the sub-block features and thumbnail features; Use the encoder to process the sub-block features and the thumbnail features to obtain the sub-block encoding features and the thumbnail encoding features respectively; Adding a sub-block identifier before and after the sub-block coding feature to obtain a sub-block unique coding feature, and adding a thumbnail identifier before and after the thumbnail coding feature to obtain a thumbnail unique coding feature; The hierarchical cross-modal attention features are formed according to the unique coding features of the sub-blocks and the unique coding features of the thumbnails, and the hierarchical cross-modal attention module is constructed; Input the training image into the hierarchical cross-modal attention module to obtain the hierarchical cross-modal attention features of the training image; The hierarchical cross-modal attention features of the training images and the encoded prompt set are concatenated to obtain concatenated features; The concatenated features are input into the feature extraction network model and then sent into the decoder to obtain the decoded output; Input the decoded output and the labels corresponding to the multi-classification task into the cross entropy function to obtain the loss function; Calculate the loss function of all categories and perform weighted summation to obtain the model loss score; During the back propagation of model parameters, the model parameters are optimized by gradient descent, so that the model loss score is reduced until the model training is completed and an image multi-classification model is obtained.

2. The method for generating a multi-classification model of an image according to claim 1, wherein: Thumbnail The calculation method is ,in is the width of the thumbnail, , is the height of the thumbnail, , is the width of the sub-block, is the scale resolution, is the block size, For rounding operation, To obtain the maximum value, To image Resize the image to the height and width of the thumbnail.

3. The method for generating a multi-classification model of an image according to claim 1, wherein: Before the step of inputting the training image into the hierarchical cross-modal attention module to obtain the hierarchical cross-modal attention features of the training image, the method further includes: A prompt set is constructed according to the number of multi-classification tasks, and then the prompts in the prompt set are encoded by an encoder to obtain an encoded prompt set.

4. An image multi-classification model generation device, characterized in that: The device comprises: An assignment module is used to assign category labels to images in a public dataset to form an image collection; A feature extraction module, used for dividing an image in the image set into a plurality of sub-blocks; Calculate the images in the image set to obtain thumbnails; The sub-block and thumbnail are respectively input into the feature extraction network model to obtain the sub-block features and thumbnail features; Use the encoder to process the sub-block features and the thumbnail features to obtain the sub-block encoding features and the thumbnail encoding features respectively; Adding a sub-block identifier before and after the sub-block coding feature to obtain a sub-block unique coding feature, and adding a thumbnail identifier before and after the thumbnail coding feature to obtain a thumbnail unique coding feature; The hierarchical cross-modal attention features are formed according to the unique coding features of the sub-blocks and the unique coding features of the thumbnails, and the hierarchical cross-modal attention module is constructed; A decoding output module is used to input the training image into the hierarchical cross-modal attention module to obtain the hierarchical cross-modal attention features of the training image; The hierarchical cross-modal attention features of the training images and the encoded prompt set are concatenated to obtain concatenated features; The concatenated features are input into the feature extraction network model and then sent into the decoder to obtain the decoded output; The weighted summation module is used to input the decoding output and the labels corresponding to the multi-classification task into the cross entropy function to obtain the loss function; Calculate the loss function of all categories and perform weighted summation to obtain the model loss score; During the back propagation of model parameters, the model parameters are optimized by gradient descent, so that the model loss score is reduced until the model training is completed and an image multi-classification model is obtained.

5. A computer device, characterized in that: The computer device includes a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the image multi-classification model generation method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which is loaded and executed by a processor to implement the image multi-classification model generation method as described in any one of claims 1-3.

Images