Image classification method and system based on large model

Abstract

The invention provides an image classification method and system based on a large model, and relates to the technical field of image classification, and the method comprises the steps: obtaining a to-be-classified modal image data set; performing local extraction on each modal image to obtain a plurality of local modal description vectors, and converting each local modal description vector into a fuzzy feature descriptor of the corresponding modal image according to the obtained modal fuzzy data; constructing a classification filter array, and for each modal image in the modal image data set, performing convolution on the modal image according to the classification filter array to obtain a modal feature domain corresponding to the modal image, and obtaining a modal feature domain corresponding to each modal image; and inputting the fuzzy feature descriptor and the modal feature domain of each modal image into an image classification model based on deep learning for classification to obtain a classification result of the modal image dataset, so that main feature information of the image data can be efficiently extracted to improve the generalization ability of the image classification model.

Description

Technical Field

[0001] The present application relates to the technical field of image classification, and more specifically, to a large model-based image classification method and system. Background Art

[0002] Image classification based on large models refers to the use of large-scale models in deep learning, such as convolutional neural networks and the latest visual transformers, to train on massive data sets to achieve automatic classification of images. Such models usually have very complex architectures, including millions or even billions of parameters, and can extract highly abstract features from images and make classification predictions based on them. Their applications cover a wide range of fields, from medical image diagnosis, visual systems for self-driving cars to image retrieval on the Internet.

[0003] Although image classification based on large models has achieved remarkable success in many tasks, it still faces a series of difficulties and challenges in practical applications. In the existing technology, although image classification models perform well on the training set, they may perform poorly when faced with new image data, especially when the data distribution changes. The large model cannot extract the main feature information of the image data, so that the image classification model may not generalize well in practical applications, resulting in a decrease in image classification accuracy. Therefore, how to efficiently extract the main feature information of image data to improve the generalization ability of image classification models is a difficult problem faced by the industry. Summary of the invention

[0004] The present application provides an image classification method and system based on a large model, which can efficiently extract the main feature information of image data to improve the generalization ability of the image classification model.

[0005] In a first aspect, the present application provides an image classification method based on a large model, the image classification method comprising the following steps: Obtain a modality image dataset to be classified; Performing local extraction on each modal image in the modal image data set respectively, thereby obtaining a local modal description vector corresponding to each modal image, and converting each local modal description vector into a fuzzy feature descriptor of the corresponding modal image according to the acquired modal fuzzy data; Constructing a classification filter array of the modal image data set, and for each modal image in the modal image data set, convolving the modal image according to the classification filter array to obtain a modal feature domain corresponding to the modal image, and further obtaining a modal feature domain corresponding to each modal image; An image classification model based on deep learning is obtained, and the fuzzy feature descriptor and the modal feature domain of each modal image are input into the image classification model based on deep learning for classification, so as to obtain a classification result of the modal image dataset.

[0006] In this embodiment, the modality image data set is a data set including a plurality of images of different modalities.

[0007] In this embodiment, locally extracting each modal image in the modal image data set to obtain a local modal description vector corresponding to each modal image specifically includes: Obtaining all local core points of each modal image in the modal image dataset; For each modal image, feature extraction is performed on all local core points of the modal image to obtain local feature descriptors corresponding to each local core point; The local modal description vector corresponding to the modal image is determined by all local feature descriptors, and then the local modal description vector corresponding to each modal image is obtained.

[0008] In this embodiment, all local core points of each modal image in the modal image dataset are obtained by extracting each modal image in the modal image dataset using a local core point monitoring algorithm, thereby obtaining all local core points of each modal image.

[0009] In this embodiment, converting each local modal description vector into a fuzzy feature descriptor of the corresponding modal image according to the acquired modal fuzzy data specifically includes: For each local modal description vector, the local modal description vector is fuzzified according to the acquired modal fuzzy data to obtain a fuzzified local modal description vector; The fuzzified local modal description vector is converted into the fuzzy feature descriptor of the corresponding modal image, and then the fuzzy feature descriptor of each modal image is obtained.

[0010] In this embodiment, converting the fuzzified local modal description vector into a fuzzy feature descriptor of the corresponding modal image is to perform norm normalization on the fuzzified local modal description vector, thereby obtaining the fuzzy feature descriptor of the corresponding modal image.

[0011] In this embodiment, constructing the classification filter array of the modality image data set specifically includes: For each modal image in the modal image data set, constructing a classification filter group corresponding to the modal image based on wavelet transform, and then obtaining a classification filter group corresponding to each modal image; A classification filter array of the modality image dataset is constructed by all classification filter groups.

[0012] In this embodiment, obtaining an image classification model based on deep learning specifically includes: Use deep learning frameworks to build image classification models; The image classification model is trained by using a pre-given training set, thereby obtaining an image classification model based on deep learning.

[0013] In this embodiment, the fuzzy feature descriptor and the modal feature domain of each modal image are input into the image classification model based on deep learning for classification, and the classification results of the modal image dataset specifically include: For each modal image, feature fusion is performed on the fuzzy feature descriptor and the modal feature domain of the modal image to obtain a modal feature fusion vector of the modal image, and then a modal feature fusion vector of each modal image is obtained; Each modality feature fusion vector is input into an image classification model based on deep learning for classification, so as to obtain the category label corresponding to each modality image, and then obtain the classification result of the modality image dataset.

[0014] In a second aspect, the present application provides a large model-based image classification system for executing a large model-based image classification method, the image classification system comprising: An initialization module is used to obtain a modality image dataset to be classified; A fuzzy feature descriptor extraction module is used to perform local extraction on each modal image in the modal image data set, thereby obtaining a local modal description vector corresponding to each modal image, and convert each local modal description vector into a fuzzy feature descriptor of the corresponding modal image according to the acquired modal fuzzy data; a modality feature domain determination module, configured to construct a classification filter array of the modality image data set, and for each modality image in the modality image data set, convolve the modality image according to the classification filter array to obtain a modality feature domain corresponding to the modality image, and further obtain a modality feature domain corresponding to each modality image; The image classification module is used to obtain an image classification model based on deep learning, input the fuzzy feature descriptor and modal feature domain of each modal image into the image classification model based on deep learning for classification, and obtain the classification result of the modal image data set.

[0015] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects: By acquiring a modal image data set to be classified; performing local extraction on each modal image in the modal image data set, thereby obtaining a local modal description vector corresponding to each modal image, and converting each local modal description vector into a fuzzy feature descriptor of the corresponding modal image according to the acquired modal fuzzy data; constructing a classification filter array of the modal image data set, for each modal image in the modal image data set, convolving the modal image according to the classification filter array, thereby obtaining a modal feature domain corresponding to the modal image, thereby obtaining a modal feature domain corresponding to each modal image; acquiring an image classification model based on deep learning, inputting the fuzzy feature descriptor and the modal feature domain of each modal image into the image classification model based on deep learning for classification, thereby obtaining a classification result of the modal image data set.

[0016] It can be seen that in the present application, the main feature information of the image data can be efficiently extracted; among them, local extraction allows the capture of details and local patterns in the modal image, which is particularly important for processing modal images with complex backgrounds or details, and by fuzzifying the local modal description vector, the fuzzy information in the modal image can be effectively processed, making the local modal description vector more robust, which helps to improve the generalization ability of the image classification model; then, by using a classification filter array, each filter can extract features of different scales in the modal image, and the modal feature domain generated by the convolution operation can retain the global information and detail information of the modal image, so that the image classification model has a more comprehensive understanding of the modal image, thereby improving the classification performance; finally, the fuzzy feature descriptor and the modal feature domain are fused and input into the image classification model, which allows the image classification model to simultaneously utilize low-level local features and high-level global features, making the image classification model more adaptable and improving the generalization ability of the image classification model on new data.

[0017] In summary, the technical solution adopted in this application can efficiently extract the main feature information of image data to improve the generalization ability of the image classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0019] Figure 1 is a flowchart of the image classification method based on a large model provided in this application; Figure 2This is a module structure diagram of the large model-based image classification system provided in this application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] The embodiment of the present application provides an image classification method and system based on a large model, the core of which is to obtain a modal image data set to be classified; perform local extraction on each modal image in the modal image data set, and then obtain a local modal description vector corresponding to each modal image, and convert each local modal description vector into a fuzzy feature descriptor of the corresponding modal image according to the acquired modal fuzzy data; construct a classification filter array of the modal image data set, and for each modal image in the modal image data set, convolve the modal image according to the classification filter array to obtain the modal feature domain corresponding to the modal image, and then obtain the modal feature domain corresponding to each modal image; obtain an image classification model based on deep learning, input the fuzzy feature descriptor and modal feature domain of each modal image into the image classification model based on deep learning for classification, and obtain the classification result of the modal image data set. The above scheme can efficiently extract the main feature information of the image data to improve the generalization ability of the image classification model.

[0022] Embodiment 1: In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 As shown in FIG. 1 , this figure is an exemplary flow chart of an image classification method based on a large model according to this embodiment of the present application, and the image classification method includes the following steps: In step S1, a modality image dataset to be classified is obtained.

[0023] In specific implementation, the modal image dataset to be classified can be uploaded to the image classification system, so that the modal image dataset to be classified can be read through the API interface. The modal image dataset is a data set containing multiple images of different modalities, and each modal image is an image of the same object obtained in different ways.

[0024] In step S2, local extraction is performed on each modal image in the modal image data set to obtain a local modal description vector corresponding to each modal image, and each local modal description vector is converted into a fuzzy feature descriptor of the corresponding modal image according to the acquired modal fuzzy data.

[0025] In this embodiment, each modal image in the modal image data set is locally extracted to obtain a local modal description vector corresponding to each modal image in the following manner: Obtaining all local core points of each modal image in the modal image dataset; For each modal image, feature extraction is performed on all local core points of the modal image to obtain local feature descriptors corresponding to each local core point; The local modal description vector corresponding to the modal image is determined by all local feature descriptors, and then the local modal description vector corresponding to each modal image is obtained.

[0026] It should be noted that, in the present application, local core points refer to local image key points in modal images; local feature descriptors are description parameters used to describe local features of modal images; and local modal description vectors are feature vectors used to describe different local features of modal images.

[0027] In the specific implementation, first, all the local core points of each modal image in the modal image data set are obtained by extracting each modal image in the modal image data set using a local core point monitoring algorithm, and then all the local core points of each modal image are obtained. The local core point monitoring algorithm used in the present application is a fast and robust feature point monitoring algorithm. Other local core point monitoring algorithms can also be used in actual implementation, which is not limited here; then, for each modal image, all the local core points of the modal image are feature extracted, and the existing feature extraction algorithm (for example: scale-invariant feature transformation algorithm) can be used to extract the features around the local core points as the corresponding local feature descriptors, so that the local feature descriptors corresponding to each local core point can be obtained; then, the local modal description vector corresponding to the modal image is determined by all the local feature descriptors, that is, all the local feature descriptors are dimensionally spliced ​​to obtain a feature vector, and the feature vector is used as the local modal description vector corresponding to the modal image. The local modal description vector corresponding to each modal image can be obtained in the above manner.

[0028] Preferably, in this embodiment, the following method can be used to convert each local modal description vector into a fuzzy feature descriptor of the corresponding modal image according to the acquired modal fuzzy data, namely: For each local modal description vector, the local modal description vector is fuzzified according to the acquired modal fuzzy data to obtain a fuzzified local modal description vector; The fuzzified local modal description vector is converted into the fuzzy feature descriptor of the corresponding modal image, and then the fuzzy feature descriptor of each modal image is obtained.

[0029] It should be noted that, in the present application, the modal fuzzy data represents the degree of blur of the local area of ​​the modal image, the local modal description vector refers to the local modal description vector after fuzzification processing; the fuzzy feature descriptor is a feature parameter used to describe the local fuzzy information of the modal image.

[0030] In the specific implementation, first, it is necessary to obtain modal fuzzy data, which can be obtained through clarity analysis of modal images, such as image gradient, spectrum characteristics or image quality evaluation methods; then, for each local modal description vector, the local modal description vector is fuzzified according to the acquired modal fuzzy data, that is, a fuzzification function can be defined to map the modal fuzzy data to the local modal description vector, thereby obtaining the fuzzified local modal description vector. In this application, fuzzification is to use the fuzzification operation in fuzzy set theory to perform nonlinear transformation on the local description vector. For example, a fuzzy weighting function can be applied to perform nonlinear transformation on the local description vector; finally, Converting the fuzzy local modal description vector into the fuzzy feature descriptor of the corresponding modal image is to perform norm normalization on the fuzzy local modal description vector, and then obtain the fuzzy feature descriptor of the corresponding modal image. In actual implementation, the norm (such as L2 norm) of the fuzzy local modal description vector is calculated. The norm is the length or size of the vector, which is usually used to measure the strength of the vector. The fuzzy local modal description vector is standardized using the calculated norm. The standardized local modal description vector can be used as the fuzzy feature descriptor of the modal image. The fuzzy feature descriptor of each modal image can be obtained in the above manner.

[0031] It should be noted that local extraction allows capturing details and local patterns in modal images, which is particularly important for processing modal images with complex backgrounds or details. In addition, by fuzzifying the local modal description vector, the fuzzy information in the modal image can be effectively processed, making the local modal description vector more robust, which helps to improve the generalization ability of the image classification model.

[0032] In step S3, a classification filter array of the modal image data set is constructed, and for each modal image in the modal image data set, the modal image is convolved according to the classification filter array to obtain a modal feature domain corresponding to the modal image, thereby obtaining a modal feature domain corresponding to each modal image.

[0033] In this embodiment, the classification filter array of the modality image data set may be constructed in the following manner: For each modal image in the modal image data set, constructing a classification filter group corresponding to the modal image based on wavelet transform, and then obtaining a classification filter group corresponding to each modal image; A classification filter array of the modality image dataset is constructed by all classification filter groups.

[0034] It should be noted that in the present application, the classification filter array is an array comprising multiple classification filters; wavelet transform is a multi-resolution decomposition technology used to analyze signals or images, which can provide localized information in both the time domain and the frequency domain. Wavelet transform can be used in image processing to extract detailed feature information such as edges and textures.

[0035] In the specific implementation, first, for each modal image in the modal image data set, a classification filter corresponding to the modal image can be constructed based on wavelet transform, that is, the modal image is decomposed into two dimensions by wavelet to obtain modal image subbands of different scales of the modal image, and different classification filters are designed according to the characteristics of the modal image subbands of each scale, so that the combination of all classification filters can be used as the classification filter group of the modal image. The classification filter group corresponding to each modal image can be obtained in the above manner; then, all the classification filter groups can be arranged in the order of the images to form a classification filter matrix, so that the classification filter matrix can be used as the classification filter array of the modal image data set.

[0036] Preferably, in this embodiment, the modality image is convolved according to the classification filter array to obtain the modality feature domain corresponding to the modality image, which can be specifically obtained in the following manner, namely: Performing two-dimensional convolution on the modal image using the corresponding classification filter group in the classification filter array to obtain a plurality of modal feature maps corresponding to the modal image; The modality feature domain corresponding to the modality image is determined through all modality feature maps.

[0037] It should be noted that, in the present application, the modal feature map represents the dot product feature map of the modal image at the corresponding scale; the modal feature domain represents the data set of all modal feature maps of the modal image.

[0038] In a specific implementation, first, a classification filter group corresponding to the current modal image is selected from the classification filter array, and the classification filter selected in the classification filter group is used to perform a two-dimensional convolution operation on the modal image. The convolution process involves sliding the filter on the modal image, and calculating the dot product between the filter and the modal image sub-region at each position to generate a modal feature map under the selected classification filter. Multiple modal feature maps of the modal image can be obtained in the above manner; then, all modal feature maps obtained by convolving the modal image are combined to form a comprehensive feature domain, that is, the modal feature domain corresponding to the modal image. In actual implementation, the modal feature maps can be fused by direct stacking, splicing, or by some weighted method, which is not limited here.

[0039] In addition, in a specific implementation, for each modal image in the modal image dataset, convolution can be performed in the manner described in the above embodiment, so as to obtain a modal feature domain corresponding to each modal image, which will not be described in detail here.

[0040] It should be noted that by using a classification filter array, each filter can extract features of different scales in the modal image, and the modal feature domain generated by the convolution operation can retain the global information (low-frequency components) and detail information (high-frequency components) of the modal image, making the image classification model have a more comprehensive understanding of the modal image, thereby improving the classification performance and helping to enhance the generalization ability of the image classification model.

[0041] In step S4, a deep learning-based image classification model is obtained, and the fuzzy feature descriptor and modal feature domain of each modal image are input into the deep learning-based image classification model for classification, so as to obtain the classification result of the modal image data set.

[0042] In this embodiment, the image classification model based on deep learning can be obtained in the following manner: Use deep learning frameworks to build image classification models; The image classification model is trained by using a pre-given training set, thereby obtaining an image classification model based on deep learning.

[0043] In the specific implementation, first, a deep learning framework is selected. The deep learning framework selected in this application is TensorFlow / Keras, which is widely used in various deep learning tasks and is particularly suitable for rapid prototyping and deployment; then, the deep learning framework is used to build an image classification model. The image classification model selected in this application is a classic convolutional neural network model; finally, a standard image classification dataset is used, and the image classification dataset is divided into a training set, a validation set, and a test set. The image classification model is trained with a pre-given training set to obtain a trained image classification model, that is, an image classification model based on deep learning is obtained.

[0044] Preferably, in this embodiment, the fuzzy feature descriptor and the modal feature domain of each modal image are input into the image classification model based on deep learning for classification, and the classification result of the modal image data set can be obtained in the following manner, namely: For each modal image, feature fusion is performed on the fuzzy feature descriptor and the modal feature domain of the modal image to obtain a modal feature fusion vector of the modal image, and then a modal feature fusion vector of each modal image is obtained; Each modality feature fusion vector is input into an image classification model based on deep learning for classification, so as to obtain the category label corresponding to each modality image, and then obtain the classification result of the modality image dataset.

[0045] In the specific implementation, first, for each modal image, the fuzzy feature descriptor and the modal feature domain of the modal image are feature fused, that is, the attention mechanism is used to dynamically adjust the weights of the fuzzy feature descriptor and the modal feature domain in the fusion process, so that the image classification model can pay more attention to the features that are most helpful for classification, thereby obtaining the modal feature fusion vector of the modal image, which refers to the feature vector of the modal image between the fuzzy feature descriptor and the modal feature domain for weighted fusion. The modal feature fusion vector of each modal image can be obtained in the above manner, and the modal feature fusion vector can more comprehensively characterize the different aspects of information of the modal image; then, the trained deep learning-based image classification model is applied to the modal feature fusion vector, that is, the modal feature fusion vector is input into the deep learning-based image classification model for classification to obtain the category label of the modal image. The category label of each modal image can be output in the above manner, thereby obtaining the classification result of the entire modal image dataset.

[0046] It should be noted that after fusing the fuzzy feature descriptor and the modal feature domain and inputting them into the image classification model, the image classification model can simultaneously utilize low-level local features and high-level global features, enhance the expressiveness of features, make the image classification model more adaptable, and improve the generalization ability of the image classification model on new data.

[0047] It can be seen that in the present application, the main feature information of the image data can be efficiently extracted; among them, local extraction allows the capture of details and local patterns in the modal image, which is particularly important for processing modal images with complex backgrounds or details, and by fuzzifying the local modal description vector, the fuzzy information in the modal image can be effectively processed, making the local modal description vector more robust, which helps to improve the generalization ability of the image classification model; then, by using a classification filter array, each filter can extract features of different scales in the modal image, and the modal feature domain generated by the convolution operation can retain the global information and detail information of the modal image, so that the image classification model has a more comprehensive understanding of the modal image, thereby improving the classification performance; finally, the fuzzy feature descriptor and the modal feature domain are fused and input into the image classification model, which allows the image classification model to simultaneously utilize low-level local features and high-level global features, making the image classification model more adaptable and improving the generalization ability of the image classification model on new data.

[0048] In summary, the technical solution adopted in this application can efficiently extract the main feature information of image data to improve the generalization ability of the image classification model.

[0049] Embodiment 2: This application provides an image classification system based on a large model, referring to Figure 2 As shown, this figure is a schematic diagram of an image classification system according to this embodiment of the present application, and the image classification system includes: Initialization module 100, used to obtain a modality image dataset to be classified; A fuzzy feature descriptor extraction module 200 is used to perform local extraction on each modal image in the modal image data set, thereby obtaining a local modal description vector corresponding to each modal image, and convert each local modal description vector into a fuzzy feature descriptor of the corresponding modal image according to the acquired modal fuzzy data; A modality feature domain determination module 300 is used to construct a classification filter array of the modality image data set, and for each modality image in the modality image data set, convolve the modality image according to the classification filter array to obtain a modality feature domain corresponding to the modality image, and then obtain a modality feature domain corresponding to each modality image; The image classification module 400 is used to obtain an image classification model based on deep learning, input the fuzzy feature descriptor and modal feature domain of each modal image into the image classification model based on deep learning for classification, and obtain the classification result of the modal image data set.

[0050] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0051] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, the storage medium including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically-erasable programmable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0052] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

Claims

1. A large model-based image classification method, characterized in that: The classification method comprises the following steps: Obtain a modality image dataset to be classified; Performing local extraction on each modal image in the modal image data set respectively, thereby obtaining a local modal description vector corresponding to each modal image, and converting each local modal description vector into a fuzzy feature descriptor of the corresponding modal image according to the acquired modal fuzzy data; Constructing a classification filter array of the modal image data set, and for each modal image in the modal image data set, convolving the modal image according to the classification filter array to obtain a modal feature domain corresponding to the modal image, and further obtaining a modal feature domain corresponding to each modal image; An image classification model based on deep learning is obtained, and the fuzzy feature descriptor and the modal feature domain of each modal image are input into the image classification model based on deep learning for classification, so as to obtain a classification result of the modal image dataset.

2. The image classification method based on a large model as claimed in claim 1, characterized in that: The modality image data set is a data set containing multiple images of different modalities.

3. The image classification method based on a large model as claimed in claim 1, characterized in that: Performing local extraction on each modal image in the modal image data set respectively, and then obtaining a local modal description vector corresponding to each modal image specifically includes: Obtaining all local core points of each modal image in the modal image dataset; For each modal image, feature extraction is performed on all local core points of the modal image to obtain local feature descriptors corresponding to each local core point; The local modal description vector corresponding to the modal image is determined by all local feature descriptors, and then the local modal description vector corresponding to each modal image is obtained.

4. The image classification method based on a large model as claimed in claim 3, characterized in that: Acquiring all local core points of each modal image in the modal image data set is to extract each modal image in the modal image data set using a local core point monitoring algorithm, thereby obtaining all local core points of each modal image.

5. The image classification method based on a large model as claimed in claim 1, characterized in that: The method of converting each local modal description vector into a fuzzy feature descriptor of the corresponding modal image according to the acquired modal fuzzy data specifically includes: For each local modal description vector, the local modal description vector is fuzzified according to the acquired modal fuzzy data to obtain a fuzzified local modal description vector; The fuzzified local modal description vector is converted into the fuzzy feature descriptor of the corresponding modal image, and then the fuzzy feature descriptor of each modal image is obtained.

6. The image classification method based on a large model as claimed in claim 5, characterized in that: Converting the fuzzified local modal description vector into a fuzzy feature descriptor of the corresponding modal image is to perform norm normalization on the fuzzified local modal description vector, and then obtain the fuzzy feature descriptor of the corresponding modal image.

7. The image classification method based on a large model as claimed in claim 1, characterized in that: Constructing the classification filter array of the modality image data set specifically includes: For each modal image in the modal image data set, constructing a classification filter group corresponding to the modal image based on wavelet transform, and then obtaining a classification filter group corresponding to each modal image; A classification filter array of the modality image dataset is constructed by all classification filter groups.

8. The image classification method based on a large model as claimed in claim 1, characterized in that: Obtaining a deep learning-based image classification model specifically includes: Use deep learning frameworks to build image classification models; The image classification model is trained by using a pre-given training set, thereby obtaining an image classification model based on deep learning.

9. The image classification method based on a large model as claimed in claim 1, characterized in that: The fuzzy feature descriptor and modal feature domain of each modal image are input into the image classification model based on deep learning for classification, and the classification results of the modal image dataset specifically include: For each modal image, feature fusion is performed on the fuzzy feature descriptor and the modal feature domain of the modal image to obtain a modal feature fusion vector of the modal image, and then a modal feature fusion vector of each modal image is obtained; Each modality feature fusion vector is input into an image classification model based on deep learning for classification, so as to obtain the category label corresponding to each modality image, and then obtain the classification result of the modality image dataset.

10. A large model-based image classification system, used to execute a large model-based image classification method according to any one of claims 1 to 9, characterized in that: The classification system includes: An initialization module is used to obtain a modality image dataset to be classified; A fuzzy feature descriptor extraction module is used to perform local extraction on each modal image in the modal image data set, thereby obtaining a local modal description vector corresponding to each modal image, and convert each local modal description vector into a fuzzy feature descriptor of the corresponding modal image according to the acquired modal fuzzy data; a modality feature domain determination module, configured to construct a classification filter array of the modality image data set, and for each modality image in the modality image data set, convolve the modality image according to the classification filter array to obtain a modality feature domain corresponding to the modality image, and further obtain a modality feature domain corresponding to each modality image; The image classification module is used to obtain an image classification model based on deep learning, input the fuzzy feature descriptor and modal feature domain of each modal image into the image classification model based on deep learning for classification, and obtain the classification result of the modal image data set.

Images