Multi-class image classification method and device based on parallel non-classification

By decomposing the multi-category image classification task into parallel single-rights and non-categorized tasks, and using multiple non-categorized models to process in parallel, the problems of low efficiency and waste of resources in the prior art are solved, and efficient and flexible images are achieved Classification method.

CN119091192APending Publication Date: 2024-12-06PAPER TECHNOLOGY (BEIJING) CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411093214.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing deep learning-based image classification model is inefficient and has severe resource waste in multi-category image classification tasks. The model has deep layers and large parameters, making it difficult to expand. Adding or replacing categories requires retraining the entire network, resulting in resources. Waste and training difficulty increase.

Method used

Using a multi-category image classification method based on parallel and non-categorized, the multi-categorized task is decomposed into multiple parallel single-categorized tasks. Each non-categorized sub-model focuses on distinguishing specific categories, and processes images to be classified in parallel through multiple non-categorized models. , reduce network depth and parameter volume, improve efficiency and scalability.

Benefits of technology

It improves the efficiency of multi-category image classification, reduces resource waste, reduces training difficulty, enhances the scalability and flexibility of the model, and avoids the need to retrain the entire network.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091192A_ABST
    Figure CN119091192A_ABST
Patent Text Reader

Abstract

The invention provides a multi-class image classification method and device based on parallel non-classification. The method comprises the following steps: acquiring an image to be classified; inputting the to-be-classified images into a plurality of trained non-classification models in parallel at the same time to obtain non-classification results output by the non-classification models; wherein the classification categories corresponding to the plurality of non-classification models are different; and determining a final image classification result of the to-be-classified image based on each non-classification result output by each non-classification model. The efficiency of multi-class image classification can be improved, and resources are saved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a multi-category image classification method and device based on parallel yes / no classification. Background Art

[0002] In the early stage of the development of image classification tasks, a series of traditional machine learning image classification methods emerged, such as K-nearest neighbor algorithm, SVM, decision tree, etc. These methods are collectively referred to as traditional machine learning image classification methods. Take SVM as an example. The full name of SVM is support vector machine. It is a common supervised learning algorithm that aims to classify input data into different categories. Its basic idea is to classify by finding a hyperplane that can separate different categories of data. The SVM algorithm was first proposed by Vapnik et al. in 1995 and has been widely used in the following decades. The SVM has the following advantages: excellent performance in processing high-dimensional data, suitable for processing complex data such as image data; can process nonlinear data and is not easy to fall into local optimal solutions; and has good generalization ability, and can also show good classification effects for small sample data sets. However, SVM has high computational complexity and long training time when processing large-scale data sets, and is sensitive to parameter selection and requires multiple parameter adjustments. In addition, SVM can generally only handle binary classification problems and is not suitable for problems with multiple categories. In addition, when processing nonlinear data such as images and videos, the kernel function needs to be tried multiple times, and improper selection will affect the classification effect.

[0003] Similar to SVM, the early traditional machine learning image classification methods all had the technical problem of being unable to complete the multi-classification task of images. Based on this, Krizhevsky et al. studied an AlexNet model in 2012. The AlexNet model won the ILSVRC 2012 competition, with a classification accuracy of more than 80%, which greatly promoted the development of image classification tasks. On this basis, a series of image classification models such as VGG, GoogLeNet, and ResNet have emerged one after another. The structure of this type of model is generally composed of a stack of convolutional layers, pooling layers, and fully connected layers, supplemented by technologies such as DropOut, BatchNormalization, and residual connections. Due to the similarity of the network structure, this type of model is collectively referred to as an image classification model based on deep learning.

[0004] Although the existing deep learning-based image classification models have shown good results in multi-classification tasks and are widely used in various application scenarios, they usually concentrate multi-classification tasks on one network model. Each category in the network is cross-connected with all other categories. This type of model has a relatively complex network structure (refer toFigure 1 ), which places certain requirements on model performance and user computing power. Specifically, on the one hand, in order to improve classification accuracy, existing deep learning-based image classification models often use the method of stacking network layers, which leads to deeper and deeper layers of the model, larger and larger parameters, and increasing training difficulty. Not only is it difficult for ordinary users and small and medium-sized enterprises to support such a huge model, but it also makes the classification efficiency poor; on the other hand, most of these image classification models concentrate multiple classification tasks on a cross-correlated network, which not only places high demands on the network model, but also makes the model not very scalable. Once an image category in a multi-classification task is replaced or a new image category is added, the model needs to be retrained, which consumes manpower and material resources and causes a waste of resources.

[0005] In summary, although the existing deep learning-based image classification model can complete the multi-classification task of images, there are problems of low efficiency and waste of resources in multi-category image classification; therefore, how to improve the efficiency of multi-category image classification and reduce the waste of resources is a technical problem that needs to be solved urgently. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a multi-category image classification method and device based on parallel non-classification to eliminate or improve one or more defects existing in the prior art.

[0007] One aspect of the present invention provides a multi-category image classification method based on parallel yes / no classification, the method comprising the following steps:

[0008] Get the image to be classified;

[0009] Inputting the image to be classified into multiple trained yes / no classification models simultaneously and in parallel to obtain the yes / no classification results output by each of the yes / no classification models; wherein the classification categories corresponding to the multiple yes / no classification models are different;

[0010] The final image classification result of the image to be classified is determined based on the respective yes / no classification results output by the respective yes / no classification models.

[0011] In some embodiments of the present invention, determining the final image classification result of the image to be classified based on each yes / no classification result output by each yes / no classification model includes:

[0012] Determine, based on each yes / no classification result output by each yes / no classification model, each representative value of the image to be classified relative to the corresponding category of each yes / no classification model;

[0013] determining an optimal representative value based on each of the representative values;

[0014] The category of the non-classification model corresponding to the optimal representative value is taken as the final image classification result of the image to be classified.

[0015] In some embodiments of the present invention, determining the optimal representative value based on the representative values ​​includes:

[0016] The maximum value or the minimum value among the plurality of representative values ​​is taken as the optimal representative value; or,

[0017] A representative value among the plurality of representative values ​​that is greater than a preset threshold or less than a preset threshold is taken as an optimal representative value.

[0018] In some embodiments of the present invention, the method further comprises:

[0019] Constructing a plurality of initial non-classification models and a sample data set of each of the initial non-classification models;

[0020] Based on the sample data sets of each of the initial yes / no classification models, each of the initial yes / no classification models is pre-trained to obtain each trained yes / no classification model.

[0021] In some embodiments of the present invention, the sample data in the sample data set includes first sample data of a category corresponding to the non-classification model and second sample data of other categories except the category corresponding to the non-classification model; and / or,

[0022] The sample data set is a sample data set after data enhancement.

[0023] In some embodiments of the present invention, the ratio of the first sample data to the second sample data is 1:1; and / or,

[0024] The non-classification model is a convolutional neural network model, a support vector machine model, a encoder-decoder model, a visual transformer model, a variational self-decoder model, a diffusion model or a generative adversarial network model.

[0025] In some embodiments of the present invention,

[0026] Before obtaining each yes / no classification result output by each yes / no classification model, the method further includes: sending the yes / no classification task corresponding to each yes / no classification model to a thread pool; allocating an active thread to each yes / no classification task so that multiple yes / no classification tasks are executed synchronously; and / or,

[0027] The multi-category image classification method includes: obtaining the image category to be expanded, determining the parameters of the to-be-expanded non-classification model corresponding to the to-be-expanded image category, and obtaining multiple expanded non-classification models based on the to-be-expanded non-classification model parameters and the multiple trained non-classification models.

[0028] Correspondingly, the present invention also discloses a multi-category image classification system based on parallel yes-no classification, including a processor, a memory and a computer program stored in the memory, wherein the processor is used to execute the computer program, and when the computer program is executed, the system implements the steps of the method described in any of the above embodiments.

[0029] According to another aspect of the present invention, a computer-readable storage medium is disclosed, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0030] According to yet another aspect of the present invention, there is provided a computer program product, comprising a computer program, which implements the steps of the method described in any of the above embodiments when executed by a processor.

[0031] The multi-category image classification method and device based on parallel yes / no classification of the present invention, when performing image classification, simultaneously and in parallel, classifies the to-be-classified images based on multiple yes / no classification models of different categories, and determines the final image classification result of the to-be-classified images according to multiple yes / no classification results output by the multiple yes / no classification models. The method decomposes the multi-classification task into multiple parallel single yes / no classification tasks, greatly reducing the depth of the network, reducing the network parameters, and reducing the difficulty of training, thereby improving the efficiency of multi-class image classification and reducing the waste of resources.

[0032] In addition, the multi-category image classification method and device can flexibly decompose, combine and expand multiple different categories of non-classification models, thereby improving the scalability of the model, thereby greatly improving the flexibility and scalability of multi-category image classification.

[0033] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0034] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention. The components in the drawings are not drawn to scale, but are only for illustrating the principles of the present invention. In order to facilitate the illustration and description of some parts of the present invention, the corresponding parts in the drawings may be enlarged, that is, they may become larger relative to other components in the exemplary device actually manufactured according to the present invention. In the drawings:

[0036] Figure 1 Schematic diagram of the architecture of the deep learning-based image classification model in the prior art.

[0037] Figure 2 4 is a flow chart of a multi-category image classification method based on parallel yes / no classification in one embodiment of the present invention.

[0038] Figure 3 It is a schematic diagram of the architecture of a non-classification model according to an embodiment of the present invention.

[0039] Figure 4 FIG. 1 is a schematic diagram of the architecture of a non-classification model according to another embodiment of the present invention.

[0040] Figure 5 It is a schematic diagram of the architecture of a multi-category image classification system based on parallel yes / no classification according to an embodiment of the present invention.

[0041] Figure 6 It is a schematic diagram of the architecture of a multi-category image classification system based on parallel non-classification after the model is expanded according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0043] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0044] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0045] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0046] Most of the existing deep learning-based image classification models concentrate multiple classification tasks on one network model. Each category in the network is cross-connected with all other categories. This not only places certain requirements on model performance and user computing power, but also makes the network itself very large. Once an image category is added or replaced in the model, the entire network needs to be retrained, which makes the scalability of this type of model poor, and the correspondence between the model and the result and the correctness and error of the classification result poorly explainable. In addition, since AlexNet, in order to improve the accuracy, the network depth and number of parameters have increased significantly, further exacerbating the problems of deep depth, large number of parameters, poor scalability and poor explainability of this type of model. In order to solve a series of problems existing in the image classification model based on deep learning in the prior art, the present application discloses a multi-category image classification method and device based on parallel yes / no classification, that is, the present application no longer places the multi-classification task on a multi-category cross-correlated network model, but decomposes the multi-classification task into multiple parallel single yes / no classification tasks, each yes / no classification sub-model focuses on distinguishing whether the input image belongs to the specific category it is responsible for, and each yes / no classification model constitutes a sub-network of its corresponding category, and does not intersect with other single yes / no sub-networks in the network structure; the method and device not only improve the efficiency of multi-category image classification, but also reduce the waste of manpower, material resources, computing power and other resources.

[0047] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0048] Figure 2 FIG. 1 is a flow chart of a multi-category image classification method based on parallel yes / no classification in one embodiment of the present invention. Figure 2 As shown, the multi-category image classification method based on parallel non-classification at least includes steps S10 to S30.

[0049] Step S10: Obtain the image to be classified.

[0050] In this embodiment, the image to be classified is an image to be classified into multiple categories, and may be an image input by a user.

[0051] Step S20: inputting the image to be classified into a plurality of trained yes / no classification models simultaneously and in parallel to obtain the yes / no classification results output by the respective yes / no classification models; wherein the classification categories corresponding to the plurality of yes / no classification models are all different.

[0052] In this step, each trained yes / no classification model receives the image to be classified in parallel at the same time, and the categories that the multiple yes / no classification models are responsible for are different, so that each yes / no classification model outputs a yes / no classification result that the image to be classified belongs to the specific category it is responsible for. In a multi-category image classification system corresponding to the multi-category image classification method, the system is mainly composed of multiple yes / no classification sub-models, and each sub-model focuses on judging whether the image to be classified belongs to the specific category it is responsible for.

[0053] In this embodiment, the specific categories to which the multiple yes / no classification models belong are different, that is, the present application decouples a single complex multi-category classification problem into multiple smaller-scale, easy-to-solve yes / no classification problems, that is, the multiple categories in the multi-classification task correspond to the categories to which the multiple yes / no classification models belong, that is, the number of yes / no classification models is the same as the number of categories in the multi-classification task. Exemplarily, when the multi-classification task includes airplane categories, car categories, bird categories, cat categories, ship categories, truck categories, and dog categories, the corresponding yes / no classification models include models for distinguishing airplanes, models for distinguishing cars, models for distinguishing birds, models for distinguishing cats, models for distinguishing ships, models for distinguishing trucks, and models for distinguishing dogs.

[0054] The multiple non-classification models in this application are combined into an overall model UniqueNet to complete the multi-category classification of the image to be classified. In addition, the types of the multiple non-classification models in the overall model UniqueNet are not limited. They can be multiple non-classification models of the same type, or multiple non-classification models of different types, or multiple non-classification models of some types that are the same and some types that are different. Exemplarily, the non-classification model can be a convolutional neural network model (CNN), a support vector machine model (SVM), a decoder-encoder model, a vision transformer model, a variational self-decoder model (VAE), a diffusion model, or a generative adversarial network model (GAN). It can be understood that the non-classification models based on convolutional neural networks and codecs listed in this embodiment are only some examples. In addition, other types of non-classification models can also be selected.

[0055] In some embodiments of the present invention, the multi-category image classification method further includes the following steps: constructing multiple initial yes / no classification models and sample data sets of each of the initial yes / no classification models; pre-training each of the initial yes / no classification models based on the sample data sets of each of the initial yes / no classification models to obtain trained yes / no classification models. In this embodiment, since the multiple yes / no classification models belong to different categories, the sample data sets used to pre-train each of the initial yes / no classification models are also different, that is, each yes / no classification model (also referred to as a yes / no classification subnetwork) has its own sample data set.

[0056] Furthermore, the sample data in the sample data set includes first sample data of a category corresponding to the non-classified model and second sample data of other categories except the category corresponding to the non-classified model. In this embodiment, the first sample data is sample data of the same category as the category to which the non-classified model belongs, and the second sample data is sample data of a different category from the category to which the non-classified model belongs; that is, the sample data in each sample data set is composed of data of the current category of the non-classified model and other categories. Optionally, the quantity ratio of the first sample data and the second sample data in the sample data set is 1:1, that is, the total quantity of sample data of the current category of the non-classified model and the total quantity of sample data of other categories in each sample data set are equal.

[0057] In addition, in order to obtain diverse data and enable the model to have better generalization ability, the data in the sample data set can be further enhanced, that is, in one embodiment, the sample data set used to pre-train the initial non-classification model is the sample data set after data enhancement. Exemplarily, data enhancement can be achieved by means of horizontal flipping, image cropping, and normalization of image pixel values; it can be understood that the data enhancement methods listed in this embodiment are only some examples, and other technical means can also be used in other embodiments to achieve data enhancement of the sample data set based on model needs.

[0058] It should be understood that the non-classification model of the present application is significantly different from other multi-classification and binary classification models. The difference is that the sample data set of the present application is constructed according to an approximately balanced ratio of "yes" and "no", where the data of the "no" class includes samples of all other classes. For example, in the non-classification data set of "cat", the samples corresponding to "yes" are "cat" class samples, and the samples corresponding to "no" include "dog", "chicken", "human", "fish", "car", "house" and other types of samples, that is, the data categories in the sample data set corresponding to the non-classification model are all-encompassing; while in the sample data set of the multi-classification model commonly used in the prior art, only samples of the model-classifiable categories are included, for example, the sample data set corresponding to the three-classification model of "cat", "dog" and "chicken" consists of three types of sample data of "cat", "dog" and "chicken". From the above content, it can be seen that the sample data set of the non-classification model of the present application is different from the sample data set corresponding to the multi-classification or binary classification model of the prior art. Therefore, the results obtained by training based on the sample data set of the present application are significantly different from the training results of the multi-classification model of the prior art in terms of the expression parameters of abstract features.

[0059] Step S30: determining a final image classification result of the image to be classified based on each yes / no classification result output by each yes / no classification model.

[0060] After obtaining the yes / no classification results output by each yes / no classification model in step S20, the final image classification result of the image to be classified is further determined based on the obtained multiple yes / no classification results. In this step, how to determine the final image classification result based on multiple yes / no classification results can be set according to actual needs; for example, the category corresponding to the maximum value among the multiple yes / no classification results can be used as the final image classification result, and the category corresponding to the minimum value among the multiple yes / no classification results can also be used as the final image classification result; in addition, a preset threshold can also be set, and the category corresponding to the value greater than or less than the preset threshold among the multiple yes / no classification results is used as the final image classification result. It can be understood that the purpose of integrating multiple yes / no classification results and selecting the category corresponding to one of the results as the final image classification result is to select the category closest to the image to be classified from the multiple yes / no classification results. Therefore, if the yes / no classification result is specifically the probability value that the image to be classified belongs to a specific category, then the category corresponding to the maximum probability value is used as the final image classification result.

[0061] In a specific embodiment, the final image classification result of the image to be classified is determined based on each yes / no classification result output by each yes / no classification model, comprising the following steps: determining each representative value of the image to be classified relative to the corresponding category of each yes / no classification model based on each yes / no classification result output by each yes / no classification model; determining an optimal representative value based on each representative value; and taking the category of the yes / no classification model corresponding to the optimal representative value as the final image classification result of the image to be classified.

[0062] The yes / no classification model is a yes / no judgment on the input image, so the yes / no classification result output by the yes / no classification model may include multiple numerical values. At this time, a value can be further selected from the multiple numerical values ​​as a representative value to represent the probability that the image to be classified belongs to the corresponding specific category. In addition, the yes / no classification result output by the yes / no classification model may also have only one numerical value, such as the difference Δ between the restored image output by the codec and the original image. At this time, the output value can be directly used as a representative value. Further, based on the multiple yes / no classification results output by the multiple yes / no classification models, multiple representative values ​​of the same number as the yes / no classification models can be determined; finally, an optimal representative value can be selected from the multiple representative values, and the category corresponding to the optimal representative value is the category to which the image to be classified belongs. In this embodiment, the optimal representative value is the representative value in the yes / no classification result output by the yes / no classification model corresponding to the category closest to the image to be classified.

[0063] When determining the optimal representative value, specifically, the maximum value or the minimum value among the multiple representative values ​​can be used as the optimal representative value; or, the representative value among the multiple representative values ​​that is greater than a preset threshold or less than a preset threshold can be used as the optimal representative value.

[0064] As described in step S20, the yes / no classification model can be a convolutional neural network model, a support vector machine model, a codec model, a visual transformer model, a variational self-decoder model, a diffusion model, and a generative adversarial network model, etc. Since the types of yes / no classification results output by various types of yes / no classification models are different, the method for determining the representative value is closely related to the type of yes / no classification model; regardless of the type of yes / no classification model, it is sufficient to ensure that the representative value is an effective representation of the probability of the image to be classified belonging to the specific category corresponding to the yes / no classification model in the yes / no classification result. For example, when the yes / no classification result output by the yes / no classification model is composed of the probability that the image to be classified belongs to the specific category and the probability that it does not belong to the specific category, the probability that the image to be classified belongs to the specific category is selected as the representative value of the yes / no classification model.

[0065] In order to further explain the relationship between the classification results, representative values ​​and optimal representative values ​​of the present application in detail, two specific examples are listed below for explanation:

[0066] Figure 3The schematic diagram of the architecture of the non-classification model of an embodiment of the present invention is shown in FIG. Figure 3 The yes / no classification models based on convolutional neural networks are combined into an overall model UniqueNet. The image to be classified is an image containing a cat, and the three yes / no classification models are pre-trained models for distinguishing specific categories of cats, dogs, and birds; in this embodiment, the image to be classified is input into the yes / no classification model based on the convolutional neural network, and after operations such as convolution, pooling, and full connection, a vector matrix with a dimension of 2×1 is obtained, and then after passing through the softmax layer, the probability values ​​of the input image output by each yes / no classification model belonging to the current class and other classes are obtained. The yes / no classification result output by the yes / no classification model is a vector matrix with a dimension of 2×1, and the two elements in the vector matrix are the probability values ​​of the image belonging to the current class and other classes (the probability value of the image belonging to other classes can also be understood as the probability value of the image not belonging to the current class). Further, the probability value of belonging to the current class in the yes / no classification result is taken as the representative value corresponding to the yes / no classification model. For example, if the image to be classified is an image containing a cat, three representative values ​​can be obtained based on three non-classification models. The three representative values ​​respectively represent the probability that the image to be classified belongs to a cat, a dog, and a bird. Finally, the maximum value of the three representative values ​​is selected as the optimal representative value. At this time, the optimal representative value represents the final image classification result obtained by the final judgment made by the overall model UniqueNet on the image to be classified.

[0067] Figure 4 Another embodiment of the present invention is a schematic diagram of the architecture of a non-classification model. In one embodiment, three Figure 4The codec-based yes / no classification model shown is combined into an overall model UniqueNet. Similar to the above embodiment, the three yes / no classification models are pre-trained models for distinguishing specific categories of cats, dogs, and birds. In this embodiment, the image to be classified is input into the codec-based yes / no classification model, encoded by the encoder, and decoded by the decoder to obtain the restored image; if the image belongs to the category that the codec is responsible for, the restored image has a small difference from the input original image, and if the image does not belong to the category that the codec is responsible for, the restored image has a large difference from the input original image. The codec-based yes / no classification model outputs not the probability that the image belongs to the current category, but the difference Δ between the restored image and the original image, that is, in the codec-based yes / no classification model, the difference Δ between the restored image and the original image is used to measure whether the input image belongs to the category that the current codec is responsible for. In this embodiment, each codec model outputs only one difference value for the yes / no classification result, and at this time, the difference value output by each codec model can be directly used as the representative value corresponding to each yes / no classification model. If the image to be classified is an image containing a cat, the difference values ​​between the three input images and the restored images are obtained based on the three encoder-decoder models: Δ 1 , Δ 2 , Δ 3 At this time, we select the minimum difference value as the optimal representative value, and the category corresponding to the minimum difference value is the final image classification result of the image to be classified.

[0068] It is understandable that the above-listed non-classification models based on convolutional neural networks and codecs are only some of the embodiments of the present application, and those skilled in the art can also select the type of non-classification model according to actual needs. In addition, a corresponding data set can be designed based on the selected type of non-classification model, and a non-classification model can be pre-trained based on the data set to save network parameters.

[0069] In addition to the above embodiments, multiple types of non-classification models can also be mixed and used. For example, in one embodiment, the overall model UniqueNet includes multiple convolutional neural network models and multiple encoder-decoder models. At this time, the multiple convolutional neural network models and the multiple encoder-decoder models each correspond to a specific category; the representative value corresponding to each convolutional neural network model is a probability value, and the representative value corresponding to each encoder-decoder model is a difference value. When determining the optimal representative value based on multiple probability representative values ​​and multiple difference representative values, the difference representative value can be first converted into a probability representative value based on a mathematical calculation method, so as to select the category to which the largest probability value among the multiple probability representative values ​​belongs as the final classification category of the image to be classified.

[0070] In summary, the multi-category image classification method of the present application can set the integration method based on the specific type of the selected yes / no classification model when integrating the yes / no classification results output by multiple yes / no classification models. For example, when all yes / no classification models are yes / no classification models based on convolutional neural networks, the category corresponding to the maximum probability value is used as the final classification result of the image to be classified; when all yes / no classification models are yes / no classification models based on codecs, the category corresponding to the minimum difference value is used as the final classification result of the image to be classified; and if multiple types of yes / no classification models are used in combination, the results output by each yes / no classification model can be first converted into probability values ​​through mathematical calculation methods, and finally the category corresponding to the maximum probability value is selected as the final classification result of the image to be classified.

[0071] In addition, in order to realize the parallel processing of multiple yes / no classification models of the present application, before obtaining the yes / no classification results output by each of the yes / no classification models, the method further includes: sending the yes / no classification tasks corresponding to each of the yes / no classification models to the thread pool; allocating an active thread to each of the yes / no classification tasks so that multiple yes / no classification tasks can be executed synchronously. When designing the thread pool, parameters such as the number of core threads and the number of active threads can be set according to the number of image categories, and each yes / no classification task can be assigned to a thread in the thread pool for processing, so that multiple yes / no classification tasks can be carried out synchronously, thereby improving classification efficiency.

[0072] Specifically, Figure 5 As shown in the figure, when the image to be classified is input into the system, the image will be transmitted to each non-classification model in parallel at the same time. Then each non-classification task will be submitted to the thread pool, and an active thread will be assigned to it to complete the non-classification task, so that multiple non-classification tasks can be carried out synchronously. When all non-classification tasks are completed, the results of each non-classification task are obtained and integrated, and finally the final classification result of the input image to be classified is given.

[0073] In addition, in order to facilitate the expansion of the yes / no classification model, during specific use, users can flexibly build the image classification system they need based on the pre-provided pre-trained yes / no classification model parameters. On the other hand, when the application scenario changes and needs to add or replace image categories, users only need to go to the pre-training library to find the network parameters of the corresponding category, or train a single yes / no classification model of the corresponding category and load it into the system.

[0074] In a specific embodiment, the multi-category image classification method further includes: obtaining the image category to be expanded, determining the parameters of the non-classification model to be expanded corresponding to the image category to be expanded, and obtaining multiple non-classification models after expansion based on the parameters of the non-classification model to be expanded and the multiple trained non-classification models. In this embodiment, the parameters of the non-classification subnetwork corresponding to each class to be added can be loaded into the multi-category image classification system based on the needs of the classification task, while the non-classification models corresponding to other categories remain unchanged. Figure 6 The schematic diagram of the architecture of the multi-category image classification system based on parallel non-classification after the model expansion of an embodiment of the present invention is shown in FIG. Figure 5 The image classification system shown in the figure is extended by Figure 5 The diagram shown here shows a non-classification model expanded with the category "dog".

[0075] Accordingly, the present invention also provides a multi-category image classification system based on parallel yes / no classification, such as Figure 5 and Figure 6 As shown in the figure, the system is built based on multiple yes / no classification models, and when pre-training the yes / no classification models, the ImageNet dataset (containing about 20,000 images of different categories) can be used as pre-training samples to train multiple yes / no classification networks, and the pre-trained network model parameters are made public. When in use, you only need to load the trained yes / no classification model parameters according to the actual task scenario, and you can freely build an image classification system that meets the current task needs, and then put it into production, which has high flexibility. If you need to add or change categories in the current scenario, you only need to find the required yes / no classification model parameters in the pre-training library, or train a new category of yes / no classification model and add it to the image classification system.

[0076] The multi-category image classification method and system of the present application is based on the core idea of ​​the divide-and-conquer algorithm, which decomposes the multi-classification task into multiple parallel and easy-to-solve non-classification tasks, so that the whole system has a small number of parameters, low training difficulty, good interpretability, and high scalability, solving the problems of deep image classification model layers, large number of parameters, and difficulty in expansion in the multi-classification model in the prior art. In addition, when the application scenario of the multi-category image classification method and system of the present application changes and it is necessary to add or replace image categories, only the newly added non-classification model needs to be trained, without retraining all models in the entire system, and the newly added non-classification sub-networks do not affect the applicability of other sub-networks, which not only greatly reduces the training cost and the waste of resources, but also greatly improves the flexibility and scalability of multi-category image classification.

[0077] Corresponding to the above method, the present invention also provides a parallel non-classified multi-category image classification system, the system includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, when the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0078] The embodiment of the present invention also provides a computer-readable storage medium and a computer program product, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0079] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0080] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0081] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-category image classification method based on parallel yes / no classification, characterized in that: The method comprises the following steps: Get the image to be classified; Inputting the image to be classified into multiple trained yes / no classification models simultaneously and in parallel to obtain the yes / no classification results output by each of the yes / no classification models; wherein the classification categories corresponding to the multiple yes / no classification models are different; The final image classification result of the image to be classified is determined based on the respective yes / no classification results output by the respective yes / no classification models.

2. The parallel non-classified multi-category image classification method according to claim 1, characterized in that: Determining a final image classification result of the image to be classified based on each yes / no classification result output by each yes / no classification model includes: Determine, based on the respective yes / no classification results output by the respective yes / no classification models, respective representative values ​​of the image to be classified relative to the corresponding categories of the respective yes / no classification models; determining an optimal representative value based on each of the representative values; The category of the non-classification model corresponding to the optimal representative value is taken as the final image classification result of the image to be classified.

3. The parallel non-classified multi-category image classification method according to claim 2, characterized in that: Determining an optimal representative value based on each of the representative values ​​includes: The maximum value or the minimum value among the plurality of representative values ​​is taken as the optimal representative value; or, A representative value among the plurality of representative values ​​that is greater than a preset threshold or less than a preset threshold is taken as an optimal representative value.

4. The parallel non-classified multi-category image classification method according to claim 1, characterized in that: The method further comprises: Constructing a plurality of initial non-classification models and a sample data set of each of the initial non-classification models; Based on the sample data sets of each of the initial yes / no classification models, each of the initial yes / no classification models is pre-trained to obtain each trained yes / no classification model.

5. The parallel non-classified multi-category image classification method according to claim 4, characterized in that: The sample data in the sample data set includes first sample data of a category corresponding to the non-classification model and second sample data of other categories except the category corresponding to the non-classification model; and / or, The sample data set is a sample data set after data enhancement.

6. The parallel non-classified multi-category image classification method according to claim 5, characterized in that: The ratio of the first sample data to the second sample data is 1:1; and / or, The non-classification model is a convolutional neural network model, a support vector machine model, a encoder-decoder model, a visual transformer model, a variational self-decoder model, a diffusion model or a generative adversarial network model.

7. The parallel non-classified multi-category image classification method according to claim 1, characterized in that: Before obtaining each yes / no classification result output by each yes / no classification model, the method further includes: sending the yes / no classification task corresponding to each yes / no classification model to a thread pool; allocating an active thread to each yes / no classification task so that multiple yes / no classification tasks are executed synchronously; and / or, The multi-category image classification method includes: obtaining the image category to be expanded, determining the parameters of the to-be-expanded non-classification model corresponding to the to-be-expanded image category, and obtaining multiple expanded non-classification models based on the to-be-expanded non-classification model parameters and the multiple trained non-classification models.

8. A multi-category image classification system based on parallel yes / no classification, comprising a processor, a memory and a computer program stored in the memory, characterized in that: The processor is used to execute the computer program. When the computer program is executed, the system implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Fruit and vegetable images classifying and identifying method and system based on model integration

    CN108319968A

  • Classification method, medium, device and calculation equipment

    CN111259932A

  • Classification model processing method and device and electronic equipment

    CN116758354A