Method and system for reorganizing flat image classification tasks into tree-shaped image classification tasks

By refactoring the flat image classification task into a tree-shaped image classification task, and utilizing pre-trained neural networks and hierarchical clustering methods, the problems of wasted computing power and insufficient modular processing in existing technologies are solved, achieving more efficient and accurate image classification.

CN118710998BActive Publication Date: 2024-11-15XIAN UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411196336.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-15
Estimated Expiration
2044-08-29

AI Technical Summary

Technical Problem

Existing image classification models train neural networks by uniformly inputting all types of images into a flattened form, which leads to wasted computing power and an inability to perform modular processing. Especially in environmental perception scenarios with limited computing resources, existing methods cannot effectively solve the problem of large differences in classification difficulty.

Method used

The flat image classification task is restructured into a tree-structured image classification task. The feature matrix is ​​extracted through a pre-trained neural network, the cosine similarity matrix and the boundary sharpness matrix are calculated, the distinguishability matrix is ​​generated, and hierarchical clustering is performed according to the pre-set distinguishability value. Similar categories are merged to construct a tree structure and gradually decompose the complex classification problem.

Benefits of technology

It reduces task complexity, improves classification accuracy and efficiency, enhances task interpretability, facilitates modular processing and optimization, and reduces the waste of computing resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118710998B_ABST
    Figure CN118710998B_ABST
Patent Text Reader

Abstract

The present application provides a method and system for reorganizing a flat image classification task into a tree-shaped image classification task, and relates to the field of image classification technology. The method comprises: extracting a feature matrix of a flat image data set by using a pre-trained neural network; calculating a cosine similarity matrix and a boundary clarity matrix based on the feature matrix; the cosine similarity matrix represents the pairwise similarity of feature vectors of all categories in the feature matrix; the boundary clarity matrix represents the degree of separation of feature vectors of different categories in the feature matrix in space; calculating a distinguishability matrix; and according to a preset distinguishability value of the distinguishability matrix, reorganizing the flat image data set in the form of a flat image classification task into the form of a tree-shaped image classification task, so as to solve the problem that the current image classification model wastes computing power and cannot perform modular processing on the classified images by uniformly inputting all types of images in a flattened form into the neural network for training.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image classification technology, and in particular to a method and system for reorganizing a flat image classification task into a tree-shaped image classification task. Background Art

[0002] With the rapid development of deep learning and computer vision technology, image classification tasks have been widely used in many fields. Typical image classification tasks require training and classifying a large amount of image data through a neural network model to achieve accurate recognition and distinction of different image categories.

[0003] When there are many types of images in a data set, the difficulty of classifying different types of images varies, and some types are easy to classify while others are difficult to classify. The existing image classification model does not consider the above factors before training, and all types of images are uniformly input into the neural network in a flattened form for training. Although the above method is simple, it has obvious disadvantages.

[0004] First, it will cause a waste of computing power. In order to obtain a higher classification accuracy, the network's feature extraction ability and model capacity must be based on images that are more difficult to classify as the main consideration, otherwise the model's classification accuracy will be relatively low. However, there are still a large number of types of images in the data to be classified that are less difficult to classify. If images that are more difficult to classify are used as the basis for network design, it means that when processing image categories that are less difficult to classify, there will be a waste of feature extraction ability and model capacity. Secondly, it is impossible to perform modular processing on the classified images. Once the neural network model training is completed, any changes to the task itself require retraining the neural network model, which is extremely time-consuming.

[0005] In some specific environmental perception scenarios, in order to quickly compile environmental and situational information, it is necessary to study efficient image classification methods. At the same time, considering the limited computing power of the computing equipment that can be carried by field operation equipment, it is necessary to study image classification methods that can save computing resources. Summary of the invention

[0006] The present application provides a method and system for reorganizing a flat image classification task into a tree-shaped image classification task, so as to solve the technical problems that the existing image classification model wastes computing power and cannot perform modular processing on the classified images by uniformly inputting all types of images in a flattened form into a neural network for training.

[0007] In a first aspect, the present application provides a method for reorganizing a flat image classification task into a tree-shaped image classification task, comprising:

[0008] A feature matrix of a flat image data set is extracted using a pre-trained neural network; the flat image data set includes: images of several different categories;

[0009] A cosine similarity matrix and a boundary definition matrix are calculated based on the feature matrix; the cosine similarity matrix represents the pairwise similarities of feature vectors of all categories in the feature matrix; the boundary definition matrix represents the degree of separation of feature vectors of different categories in the feature matrix in space;

[0010] Calculating a distinguishability matrix based on the cosine similarity matrix and the boundary clarity matrix;

[0011] According to the preset distinguishability value of the distinguishability matrix, the flat image data set is reorganized in the form of a flat image classification task into the form of a tree image classification task.

[0012] In some embodiments, calculating the cosine similarity matrix based on the feature matrix includes:

[0013] Based on the feature matrix, extracting first image features of a first type of image; the first type of image is an image of any category in the flat image data set;

[0014] Based on the first image feature, calculating an average value of the first image feature to obtain a first category center;

[0015] Based on the feature matrix, extracting second image features of a second type of image; the second type of image is an image of any category in the flat image data set;

[0016] Based on the second image feature, calculating an average value of the second image feature to obtain a second category center;

[0017] A cosine similarity matrix is ​​obtained by using a cosine similarity formula according to the first category center and the second category center corresponding to all category images in the flattened image data set.

[0018] In some embodiments, the cosine similarity matrix is ​​determined by the following formula:

[0019] ;

[0020] In the formula, A k is a Category I centre; B k A Category II centre; i +1 represents the number of rows in the cosine similarity matrix; j +1 represents the number of columns in the cosine similarity matrix; 0≤ k ≤ n-1.

[0021] In some embodiments, the calculating the boundary definition matrix based on the feature matrix comprises:

[0022] Based on the feature matrix, a first data feature and a second data feature are obtained according to the data label; the first data feature and the second data feature are data features corresponding to any two category images in the flat image data set; the data label is a classification text of each category image in the flat image data set;

[0023] Using the first data feature and the second data feature to train a support vector machine model, and obtaining an average distance from a support vector to a decision boundary and the number of support vectors;

[0024] A boundary sharpness matrix is ​​obtained based on the average distance of the support vector to the decision boundary and the number of support vectors.

[0025] In some embodiments, the boundary definition matrix is ​​determined by the following formula:

[0026] ;

[0027] Where SVM_d is the average distance from the support vector to the decision boundary; SVM_n is the number of support vectors; i +1 represents the number of rows in the boundary sharpness matrix; j +1 represents the number of columns in the boundary sharpness matrix.

[0028] In some embodiments, the distinguishability matrix is ​​determined by the following formula:

[0029] ;

[0030] In the formula, similarity _ matrix is the cosine similarity matrix; clarity _ matrix is the boundary definition matrix.

[0031] In some embodiments, the reorganizing the flat image dataset in the form of a flat image classification task into a tree image classification task according to the preset distinguishability value of the distinguishability matrix comprises:

[0032] Define the preset value of distinguishability;

[0033] Based on the distinguishability matrix, obtaining a first cluster and a second cluster in the distinguishability matrix; the distance between the first cluster and the second cluster is the smallest compared to other clusters in the distinguishability matrix;

[0034] Determine whether the distance between the first cluster and the second cluster is less than or equal to the preset distinguishability value, and if so, merge the first cluster and the second cluster to generate an intermediate cluster;

[0035] According to the intermediate clusters and the unmerged clusters, the flat image dataset is reorganized in the form of a flat image classification task into the form of a tree image classification task.

[0036] In some embodiments, the step of reorganizing the flat image dataset in the form of a flat image classification task into a tree image classification task according to the intermediate clusters and the unmerged clusters comprises:

[0037] Calculating the distinguishability distance between the intermediate clusters;

[0038] Based on the distinguishability distance, the distinguishability matrix is ​​updated, and the step of determining whether the distance between the first cluster and the second cluster is less than or equal to the preset distinguishability value is repeatedly performed until the distance between the first cluster and the second cluster in the distinguishability matrix is ​​greater than the preset distinguishability value, thereby generating a target cluster;

[0039] According to the first cluster, the second cluster, the intermediate cluster, the target cluster, and the unmerged cluster, the flat image dataset is reorganized in the form of a flat image classification task into the form of a tree image classification task.

[0040] In some embodiments, the distinguishability distance between the intermediate clusters is determined by the following formula:

[0041] ;

[0042] In the formula, C ij is the middle cluster; C i is the first cluster; C j For the second cluster; k 0 Indicates other clusters that are not currently merged;

[0043] ;

[0044] .

[0045] A second aspect of the present application provides a system for reorganizing a flat image classification task into a tree-shaped image classification task, comprising:

[0046] An extraction module, wherein the extraction module is configured to: extract a feature matrix of a flat image data set using a pre-trained neural network; the flat image data set includes: images of several different categories;

[0047] A first calculation module, wherein the first calculation module is configured to: calculate a cosine similarity matrix and a boundary clarity matrix based on the feature matrix; the cosine similarity matrix represents the pairwise similarities of feature vectors of all categories in the feature matrix; the boundary clarity matrix represents the degree of separation of feature vectors of different categories in the feature matrix in space;

[0048] A second calculation module, the second calculation module is configured to: calculate a distinguishability matrix based on the cosine similarity matrix and the boundary clarity matrix;

[0049] A reorganization module is configured to: reorganize the flat image data set in the form of a flat image classification task into a tree image classification task according to a preset distinguishability value of the distinguishability matrix.

[0050] The present application provides a method and system for reorganizing a flat image classification task into a tree-shaped image classification task, the method comprising: extracting a feature matrix of a flat image data set using a pre-trained neural network; calculating a cosine similarity matrix and a boundary clarity matrix based on the feature matrix; the cosine similarity matrix represents the pairwise similarities of feature vectors of all categories in the feature matrix; the boundary clarity matrix represents the degree of separation of feature vectors of different categories in the feature matrix in space; calculating a distinguishability matrix based on the cosine similarity matrix and the boundary clarity matrix; and reorganizing the flat image data set in the form of a flat image classification task into the form of a tree-shaped image classification task according to a preset distinguishability value of the distinguishability matrix, so as to reduce task complexity, improve the accuracy and reliability of the solution, enhance the interpretability of the task, and facilitate modular processing and optimization through the tree-shaped image classification task. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0052] Figure 1 A flowchart of a method for reorganizing a flat image classification task into a tree-shaped image classification task in this application;

[0053] Figure 2 A framework diagram of the method for reorganizing the flat image classification task into a tree-shaped image classification task in this application;

[0054] Figure 3 A complete framework diagram for automatically reorganizing the flat task into a tree-shaped classification task in this application;

[0055] Figure 4 Reorganize the flat image classification task in this application into a tree classification task flow chart;

[0056] Figure 5 Schematic diagram of the reorganization of the flat image classification task into a tree classification task in this application. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0058] In some technologies, the image classification model uniformly inputs all types of images into the neural network in a flat form for training, which will cause a waste of computing power and fail to perform modular processing on the classified images. In order to solve this technical problem, the present application provides a method and system for reorganizing a flat image classification task into a tree-shaped image classification task. The method and system for reorganizing a flat image classification task into a tree-shaped image classification task are described below:

[0059] For example, with the rapid development of deep learning and computer vision technology, image classification tasks have been widely used in many fields. Typical image classification tasks require training and classification of a large amount of image data through a neural network model to achieve accurate recognition and distinction of different image categories. However, when there are many types of pictures in a data set, the difficulty of classifying different types of pictures inside it varies, and some types are easy to classify and some types are difficult to classify. The existing image classification model does not consider this factor before training. The existing method uniformly inputs all types of pictures into the neural network in a flattened form for training. Although this approach is simple, it has obvious disadvantages.

[0060] First, it will cause a waste of computing power. In order to obtain a higher classification accuracy, the network's feature extraction capability and model capacity must be based on the highest distinguishability value (the higher the distinguishability value, the more difficult it is to classify the item), otherwise the model's classification accuracy will be relatively low. However, there are still a large number of types of images in the data to be classified that have low distinguishability scores. If the maximum distinguishability value is used as the basis for network design, it means that when processing categories with smaller distinguishability values ​​(smaller distinguishability values ​​mean easier classification), there will be a waste of feature extraction capabilities and model capacity. Secondly, it is impossible to perform modular processing on the classified images. Once the neural network model training is completed, any changes to the task itself require retraining the neural network model.

[0061] In some specific environmental perception scenarios, in order to quickly compile environmental and situational information, it is necessary to study efficient image classification methods. At the same time, considering the limited computing power of the computing equipment that can be carried by field operation equipment, it is necessary to study image classification methods that can save computing resources.

[0062] In response to the above problems, the present application proposes a method and system for reorganizing a flat image classification task into a tree-shaped image classification task, which automatically reorganizes the original flat classification task into a multi-layer classification task with a tree structure. The automatic reorganization process is as follows: First, we identify the categories that are more difficult to classify and cluster these categories into a new parent node; the parent node represents a new category that does not exist in the original data set, but is the result of summarizing the original categories that are difficult to distinguish. This node contains all these difficult-to-distinguish categories. Next, the parent node is combined with other categories that are not involved in the clustering to form a new classification problem. This process is repeated, and each time the difficult-to-distinguish categories are further summarized and clustered until a complete tree structure is constructed. Through the above method, we gradually decompose the classification task into multiple levels, and process the categories that are more difficult to classify step by step, thereby effectively simplifying complex classification problems and improving the accuracy and efficiency of classification.

[0063] Depend on Figure 1 It can be seen that the first aspect of the present application provides a method for reorganizing a flat image classification task into a tree-shaped image classification task, comprising the following steps:

[0064] S100: Extract the feature matrix of a flat image data set using a pre-trained neural network; the flat image data set includes: a number of images of different categories; in the embodiment of the present application, the cifar100 data set is taken as an example, that is, the flat image data set is the cifar100 data set; the cifar100 data set has 100 different categories and a total of 50,000 images, and the pre-trained resnet50 (resnet50 is the pre-trained neural network) is used to extract the features of the training set images, and the output of the second-to-last layer of resnet50 is taken as the feature to obtain the training set data features. The feature extraction method of the above method obtains the feature matrix of the cifar100 data set, and its shape is [50000, 2048], where 50000 represents the number of images, and 2048 represents the feature vector dimension of each image.

[0065] For example, the present application embodiment proposes a concept: the distinguishability of an image, which consists of two parts: one is the feature center similarity between categories, and the other is the boundary clarity of features between categories. The following method is the definition and detailed derivation of the feature center similarity between categories and the boundary clarity of features between categories.

[0066] S200: Calculating a cosine similarity matrix and a boundary clarity matrix based on the feature matrix; the cosine similarity matrix represents the pairwise similarities of feature vectors of all categories in the feature matrix; the boundary clarity matrix represents the degree of separation of feature vectors of different categories in the feature matrix in space.

[0067] Exemplarily, the feature center similarity between the categories refers to the similarity between the feature vector centers of different categories. The higher the similarity, the more difficult it is to distinguish the features of the two categories. Due to the computational complexity, the embodiment of the present application uses cosine similarity to evaluate the feature similarity between categories.

[0068] The calculation of the cosine similarity matrix based on the feature matrix comprises the following sub-steps:

[0069] S210: Based on the feature matrix, extract the first image feature of the first category of images; the first category of images is an image of any category in the flat image data set; extract all image features of the i-th category of images from the training features, a total of 500; (0≤i≤99).

[0070] S220: Based on the first image feature, calculate the average value of the first image feature to obtain a first category center; calculate the average value of 500 features of the i-th category image to obtain a first category center (class_center(i)), where the first category center is a vector of 1×2048 dimensions.

[0071] S230: Based on the feature matrix, extract second image features of the second category of images; the second category of images is an image of any category in the flat image data set; extract all image features of the j-th category of images from the training features, a total of 500; (0≤j≤99).

[0072] S240: Based on the second image feature, calculate the average value of the second image feature to obtain a second category center; calculate the average value of 500 features of the j-th category image to obtain a second category center (class_center(j)), where the second category center is a vector of 1×2048 dimensions.

[0073] S250: According to the first category center and the second category center corresponding to all category images in the flattened image data set, a cosine similarity formula is used to obtain a cosine similarity matrix. The cosine similarity matrix is ​​determined by the following formula:

[0074] ;

[0075] In the formula, A k is the first class center (class_center(i)); B k is the second category center (class_center(j)); i +1 represents the number of rows in the cosine similarity matrix; j +1 represents the number of columns in the cosine similarity matrix; 0≤ k ≤ n -1.

[0076] Exemplarily, by traversing the categories of all training set features with the first category center and the second category center, the pairwise similarities of all training set feature categories can be obtained, which are represented by a cosine similarity matrix:

[0077] Similatiry_matrix[i+1,j+1];st i, j = [0, 99];

[0078] The cosine similarity matrix is ​​a symmetric matrix; the i+1th row and j+1th column of the cosine similarity matrix are the cosine similarities of the centers of the first and second categories. (The reason why i and j are added by 1 is that the serial number of the cosine similarity matrix starts from 1, while the serial number of i and j data starts from 0); the larger the value of Similatiry_matrix[i+1, j+1], the higher the similarity of the feature centers between categories i and j, that is, the more difficult it is to distinguish, and thus the more difficult it is to classify.

[0079] Exemplarily, the boundary clarity of features between categories refers to the degree of separation of feature vectors of different categories in space; the clearer the boundary, the clearer the feature vectors are in space and easier to classify. Because the required data is computable quantitative data and can be obtained using low-complexity methods before the model training stage, the embodiment of the present application uses a method based on support vector machines to analyze the clarity of category boundaries. Among them, in the embodiment of the present application, a 100×100 boundary clarity matrix clarity_matrix is ​​generated for 100 categories of pictures to represent the boundary clarity of each pair of categories. The boundary clarity matrix is ​​calculated using the following method.

[0080] The step of calculating the boundary definition matrix based on the feature matrix comprises the following sub-steps:

[0081] S260: Based on the feature matrix, according to the data label, a first data feature and a second data feature are obtained; the first data feature and the second data feature are data features corresponding to any two categories of images in the flat image dataset; the data label is the classification text of each category of images in the flat image dataset; according to the data label, a first data feature data_features(i) and a second data feature data_feature(j) are obtained. Where i and j represent any two categories in the flat image dataset.

[0082] Exemplarily, in the context of machine learning or deep learning, a data label (also called a class label or target variable) refers to information associated with each sample in a dataset, which indicates the category or attribute to which the sample belongs. For a flat image dataset, which can be used directly for model training, the classification text / data label of each category of images is used to identify the specific category to which these images belong. Specifically, suppose we have a flat image dataset that contains pictures of various animals, such as cats, dogs, birds, etc. The classification text / data labels of the above images will be: cat, dog, bird; for each picture in the dataset, there will be a corresponding label indicating which of the three categories the picture belongs to. For example, a picture of a cat will have a label "cat", a picture of a dog will have a label "dog", and so on. When training a machine learning model, these labels are the goals of model learning, that is, the model needs to learn how to predict its corresponding label (i.e., category) based on the content of the image (such as color, texture, shape, etc.).

[0083] S270: Use the first data feature and the second data feature to train a support vector machine model to obtain an average distance from a support vector to a decision boundary and the number of support vectors; use a radial basis function network (RBF) to train a support vector machine (SVM) model to obtain an average distance of support vectors and the amount of support vector data, represented by SVM_d and SVM_n, respectively. If the number of support vectors is small, it means that the boundary is clearer, and if the average distance from the support vector to the decision boundary is large, it means that the boundary is clearer.

[0084] S280: Based on the average distance from the support vector to the decision boundary and the number of support vectors, a boundary clarity matrix is ​​obtained. The boundary clarity matrix is ​​determined by the following formula:

[0085] ;

[0086] Where SVM_d is the average distance from the support vector to the decision boundary; SVM_n is the number of support vectors; i +1 represents the number of rows in the boundary sharpness matrix; j +1 represents the number of columns in the boundary sharpness matrix.

[0087] Exemplarily, i+1 and j+1 represent the i+1th row and j+1th column of the boundary clarity matrix, respectively; i and j are added by 1 because the sequence number of the boundary clarity matrix starts from 1, while the sequence number of i and j data starts from 0. Through the above method, the boundary clarity of any two types of pictures i and j is obtained, and then the boundary clarity matrix clarity_matrix is ​​obtained. This matrix is ​​a symmetric matrix; the smaller the value of clarity_matrix[i+1, j+1], the less clear the boundary between categories i and j is, that is, the more difficult it is to classify.

[0088] S300: Calculate a distinguishability matrix based on the cosine similarity matrix and the boundary definition matrix; the distinguishability matrix is ​​determined by the following formula:

[0089] ;

[0090] In the formula, similarity _ matrix is the cosine similarity matrix; clarity _ matrix is the boundary definition matrix.

[0091] For example, the distinguishability matrix is ​​defined as D=distinguish_matrix, which is a 100×100 matrix that represents the distinguishability between two categories. According to the meaning of similarity_matrix (cosine similarity matrix) and clarity_matrix (boundary clarity matrix), the larger the value of the distinguishability matrix, the more difficult it is to distinguish between categories i and j. Because both the cosine similarity matrix and the boundary clarity matrix are symmetric matrices, D is also a symmetric matrix, that is, D i+1,j+1 =D j+1,i+1 The quantified value of the distinguishability between the image categories can be obtained through the distinguishability matrix.

[0092] S400: According to the preset distinguishability value of the distinguishability matrix, the flat image data set is reorganized in the form of a flat image classification task into a tree image classification task.

[0093] like Figure 2 As shown, the present application provides a method for reorganizing a flat image classification task into a tree-shaped image classification task, which is used to optimize the image classification task, improve the classification accuracy and reduce the waste of computing resources. Specifically, the present application includes the following steps: First, the features of the training set images are extracted using a pre-trained neural network to obtain a feature matrix. The second step is to define the ease of classification of the image, including the feature center similarity between categories and the boundary clarity of the features between categories. Through the feature center similarity and boundary clarity, an easy-to-distinguish matrix is ​​generated. The third step is to automatically reorganize the flat image classification task into a tree-shaped classification task through the easy-to-distinguish matrix. The present application uses a hierarchical clustering method using a preset easy-to-distinguish interval as a judgment criterion. By gradually merging similar categories, a hierarchical clustering tree diagram is constructed. The advantage of the present application is that by quantifying the easy-to-distinguish between categories, the flat classification task is intelligently reorganized into a tree-shaped classification task, thereby solving the problem of resource waste and low accuracy of existing methods when dealing with categories with large differences in classification difficulty.

[0094] Exemplary, a method for automatically reorganizing a flat image classification task into a tree classification problem based on distinguishability, the specific steps are as follows: Taking the cifar100 dataset as an example, the distinguishability matrix D represents the distinguishability of 100 types of images. The larger the value of row i+1 and column j+1 in D, the more difficult it is to distinguish between classes i and j, and the more common features classes i and j have. A network with greater feature extraction capability and model capacity is needed to classify them. This difficult-to-distinguish task is defined as a difficult task. When there are many types of images in a dataset, the difficulty of classifying different types of images within it is different. Some types are easy to classify, while some types are difficult to classify. That is, there will be a large range of values ​​in D, for example, D i+1,j+1=0.95, D i+1,k+1 =0.3, the former is obviously greater than the latter (the former is more difficult to classify than the latter). The existing image classification model does not consider the above factors before training. The existing method uniformly inputs all types of pictures into the neural network in a flattened form for training. There are the following disadvantages:

[0095] Waste of computing power: In order to obtain a higher classification accuracy, the network's feature extraction capability and model capacity must be increased by D i+1,k+1 The largest value in D is taken as the main consideration, otherwise the classification accuracy of the model will be relatively low. However, there are still a large number of types of images in the data to be classified with low distinguishability scores. If Max(D) (representing the largest value in D) is used as the basis for network design, it means that when processing categories with smaller D, there will be a waste of feature extraction capabilities and model capacity. It is impossible to perform modular processing on classified images: once the neural network model training is completed, any changes to the task itself require retraining the neural network model.

[0096] In response to the above shortcomings, the present application proposes a method for reorganizing a flat image classification task into a tree-shaped image classification task; specifically, the original flat classification task is automatically reorganized into a multi-layer classification task with a tree structure, and the automatic reorganization process is as follows: First, identify the categories that are more difficult to classify, and cluster these categories into a new parent node. This parent node represents a new category that does not exist in the original data set, but is the result of summarizing the original categories that are difficult to distinguish. This node contains all these difficult-to-distinguish categories. Next, this new parent node is combined with other categories that are not involved in the clustering to form a new classification problem. This process is repeated, and each time the difficult-to-distinguish categories are further summarized and clustered until a complete tree structure is constructed. Figure 5 shown.

[0097] Exemplarily, through the above method, we gradually decompose the classification task into multiple levels and process the categories with higher classification difficulty step by step, thereby effectively simplifying the complex classification problem and improving the accuracy and efficiency of classification.

[0098] like Figure 5As shown in the figure, taking the five categories A, B, C, D, and E as an example, the current reorganization method of image classification tasks is a flat structure. For example, there is only one layer of structure for the five categories of images A, B, C, D, and E, while the tree structure is a three-layer structure. The specific reorganization method is as follows: If A and B are difficult to distinguish, D and E are difficult to distinguish, A, B, D, E, and C are relatively easy to distinguish, A and B are easy to distinguish D, and A and B are easy to distinguish E, then A and B are clustered to form a new category M, and D and E are clustered into a new category N. Subsequently, parent class nodes M and N are generated at the top level of the tree structure, and the original category C is retained. This reorganization transforms the original flat classification problem into two sub-problems: first, classify C, M, and N. When classifying, first determine which category the image belongs to among categories C, M, and N; if the image belongs to category C, the classification task ends. If the image belongs to category M, then further determine whether it is A or B. If the image belongs to category N, then further determine whether it is D or E. After the above process, a flat classification task is reorganized into a tree-like classification task. The beneficial effects of the above method are:

[0099] Reduce task complexity: Decomposing complex classification tasks into multiple smaller subtasks helps simplify the problem. The processing difficulty and complexity of each subtask are relatively low, making the overall problem easier to manage and solve. Improve the accuracy and reliability of the solution: Smaller subtasks can handle classification problems more focused and meticulously, thereby improving the accuracy of classification. As the complexity of each subtask is reduced, the models and methods required to solve these subtasks may be more concise and effective, thereby improving the reliability of the overall solution. Enhance the interpretability of tasks: By decomposing complex tasks into multiple subtasks, the processing process of each subtask and its role in the overall task can be more clearly understood and explained. The above decomposition and association process helps to improve the transparency and interpretability of the overall model. Facilitate modular processing and optimization: After the task is decomposed, each subtask can be optimized and processed separately, thereby gradually improving the performance of the overall solution. The above modular processing method also facilitates independent adjustment and optimization of a specific subtask when necessary without affecting other parts.

[0100] The step of reorganizing the flat tasks in the flat image dataset into a tree-shaped image classification task according to the preset distinguishability values ​​of the distinguishability matrix comprises the following sub-steps:

[0101] S410: Defining a preset value for easy distinguishability ; One purpose of automatically reorganizing flat tasks into tree-shaped classification tasks is to put categories with similar classification difficulty under the same parent node and use a neural network to align the classification, so as to maximize the performance of the neural network. If the classification difficulty is different, it will cause a waste of neural network performance. Based on the above problems, the preset value of easy distinguishability is defined, indicating that if the difference in easy distinguishability between different categories is within the range, then the difficulty of these classifications is considered to be close. The specific definition is as follows:

[0102] For classification tasks with multiple categories, Max(D)-Min(D)< , then it is considered that the classification difficulty of all categories in the current D is close. Among them, The value is set by experience. i,j =0.85, D j,k =0.88, D k,m =0.7, if Then D i,j -D j,k < , then the classification difficulty of i, j, and k is close, but the classification difficulty of m is greater.

[0103] Exemplarily, by artificially giving a numerical value of the preset value of easy distinguishability, a tree structure diagram of the classification task is further constructed. In order to automatically reorganize the flat task into a tree classification task, the present application uses a hierarchical clustering method to construct a tree structure diagram. The reason for selecting hierarchical clustering is that hierarchical clustering not only considers the distance between data points, but also can control the distance between categories by cutting the tree diagram.

[0104] Exemplarily, this application uses the distinguishable preset value As the judgment criterion of hierarchical clustering, the specific clustering process is as follows: each target detection category feature vector is regarded as an independent cluster. Initially, assuming that there are n target detection categories, there are n independent clusters, each cluster contains a feature vector of a target detection category. Based on the feature vector of each cluster, the distinguishability matrix between all clusters is generated.

[0105] S420: Based on the distinguishability matrix, obtain a first cluster and a second cluster in the distinguishability matrix; the distance between the first cluster and the second cluster is the smallest compared to other clusters in the distinguishability matrix; find two clusters with the smallest distance in the distinguishability matrix: find two clusters with the smallest distance in the distinguishability matrix, record them as clusters C i and Cluster C j .

[0106] S430: Determine whether the distance between the first cluster and the second cluster is less than or equal to the preset distinguishability value. If so, merge the first cluster and the second cluster to generate an intermediate cluster. C i and Cluster C j Is the distance within the preset value of easy distinction: If the cluster C i and Cluster C j The distance between them is less than or equal to the preset value of easy distinction , then merge these two clusters into a new cluster C ij ; If the cluster C i and Cluster C j The distance between them is greater than the preset value of easy distinction , they are considered not to belong to the same category and are not merged.

[0107] S440: According to the intermediate clusters and the unmerged clusters, the flat image dataset is reorganized in the form of a flat image classification task into a tree image classification task.

[0108] The step of reorganizing the flat tasks in the flat image dataset into a tree-shaped image classification task according to the intermediate clusters and the unmerged clusters comprises the following sub-steps:

[0109] S441: Calculate the distinguishability distance between the intermediate clusters; the distinguishability distance between the intermediate clusters is determined by the following formula:

[0110] ;

[0111] In the formula, C ij is the middle cluster; C i is the first cluster; C j For the second cluster; k 0 Indicates other clusters that are not currently merged;

[0112] ;

[0113] .

[0114] Among them, the above parameter values ​​can refer to the formulas of the above cosine similarity matrix and boundary clarity matrix, which will not be described in detail here.

[0115] S442: Based on the distinguishability distance, update the distinguishability matrix, and repeat the step of determining whether the distance between the first cluster and the second cluster is less than or equal to the distinguishability preset value, until the distance between the first cluster and the second cluster in the distinguishability matrix is ​​greater than the distinguishability preset value, and generate a target cluster; repeat the above hierarchical clustering steps until no more merging can be performed, that is, the distance between all clusters is greater than the distinguishability preset value At this point, it is considered that all image types corresponding to the node are easy to distinguish, and the process of reorganizing into a tree structure ends.

[0116] S443: According to the first cluster, the second cluster, the intermediate cluster, the target cluster, and the unmerged cluster, the flat image data set is reorganized in the form of a flat image classification task into a tree image classification task. A dendrogram is constructed based on the cluster merging history generated by the above-mentioned hierarchical clustering process. Each branch in the dendrogram represents a cluster merging process, and the height of the tree represents the distance between clusters. It can be understood that each cluster corresponds to an image of a certain category. When the distance between two clusters is less than or equal to the preset value of easy distinguishability When , the branches merge; otherwise, the branches separate.

[0117] like Figure 3 As shown, the present application provides a method for reorganizing a flat image classification task into a tree-shaped image classification task. First, a pre-trained neural network is used to extract features of a training set image. Then, the feature center similarity and feature boundary clarity between categories are calculated. Then, the feature center similarity and feature boundary clarity between categories are used to calculate the distinguishability matrix between categories. Next, a distinguishability preset value is defined. A hierarchical clustering method is used to judge the first cluster and the second cluster in the distinguishability matrix against the distinguishability preset value. If the first cluster and the second cluster are greater than the distinguishability preset value, they are merged. Figure 5 As shown, taking the five categories A, B, C, D, and E as an example, if the distance between the clusters representing category A and category B is less than or equal to the preset value of easy distinction , then merge category A and category B. If the distance between category D and category E representing the cluster is less than or equal to the preset value of easy distinction , then merge category D and category E; and retain the original category C; then category M and category N are the merged intermediate clusters, and category C is the unmerged cluster; then recalculate the easy-to-distinguish matrix of the merged intermediate cluster, that is, category M and category N represent the easy-to-distinguish matrix of the cluster, and determine the distance and easy-to-distinguish preset value between category M and category N If the distance between category M and category N is less than or equal to the preset value of easy distinction , then continue to merge them. If the distance between category M and category N is greater than the preset value of easy distinction , no more merging is performed, and category M, category N, and category C are the target clusters. The cluster merging history generated by the above hierarchical clustering process is used to construct a dendrogram; that is, the process of reorganizing the flat task in the flat image dataset into a tree-shaped image classification task is completed.

[0118] Exemplarily, the present application inputs a flat image data set into a reorganization model, and then inputs a pre-trained neural network into the reorganization model, and uses the pre-trained neural network to extract a feature matrix of the flat image data set; based on the feature matrix, a cosine similarity matrix and a boundary clarity matrix are calculated through the reorganization model; based on the cosine similarity matrix and the boundary clarity matrix, a distinguishability matrix is ​​calculated using the reorganization model; finally, according to a preset distinguishability value of the distinguishability matrix, the flat image data set is reorganized from a flat image classification task into a tree image classification task, and input into a display device, wherein the reorganization model can be an electronic device such as a computer.

[0119] Exemplarily, the final dendrogram shows the hierarchical relationship and similarity of the target detection categories, e.g. Figure 4 As shown. By analyzing the dendrogram, the classification and distinction between different categories can be intuitively identified and explained. The specific analysis methods are as follows: Identify hierarchical relationships: The hierarchical structure of the dendrogram reflects the classification relationship between each target detection category. The closer the branch is to the root node, it means that these categories were merged at an earlier stage, that is, their similarity is higher. Distinguish similar categories: Close branches in the dendrogram indicate that the similarity between these categories is high and difficult to distinguish. By observing the height of the branch, you can understand the difficulty of distinguishing between different categories. Optimize the classification model: Based on the hierarchical structure of the dendrogram, the classification model can be optimized. For example, for categories with high similarity, a more complex model can be used for distinction; for categories with low similarity, a simpler model can be used to improve classification efficiency and accuracy, and reduce the waste of computing resources.

[0120] A second aspect of the present application provides a system for reorganizing a flat image classification task into a tree-shaped image classification task, comprising: an extraction module, wherein the extraction module is configured to: extract a feature matrix of a flat image data set using a pre-trained neural network;

[0121] The first calculation module is configured to: calculate the cosine similarity matrix and the boundary clarity matrix based on the feature matrix; the cosine similarity matrix represents the pairwise similarity of all category feature vectors in the feature matrix; the boundary clarity matrix represents the degree of separation of feature vectors of different categories in the feature matrix in space; the second calculation module is configured to: calculate the distinguishability matrix based on the cosine similarity matrix and the boundary clarity matrix; the reorganization module is configured to: reorganize the flat image data set in the form of a flat image classification task into the form of a tree image classification task according to the preset distinguishability value of the distinguishability matrix. The effects of each part of the above-mentioned system embodiment during operation can refer to the effects of the above-mentioned method embodiment, which will not be repeated here.

[0122] The above specific implementation methods further explain in detail the purpose, technical solutions and beneficial effects of the embodiments of the present application. It should be understood that the above are only specific implementation methods of the embodiments of the present application and are not used to limit the protection scope of the embodiments of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the protection scope of the embodiments of the present application.

Claims

1. A method for reorganizing a flat image classification task into a tree-shaped image classification task, characterized in that: include: Extract feature matrices from flattened image datasets using pre-trained neural networks; The flat image dataset includes: a number of images of different categories; A cosine similarity matrix and a boundary definition matrix are calculated based on the feature matrix; the cosine similarity matrix represents the pairwise similarities of feature vectors of all categories in the feature matrix; the boundary definition matrix represents the degree of separation of feature vectors of different categories in the feature matrix in space; Based on the cosine similarity matrix and the boundary definition matrix, a distinguishability matrix is ​​calculated; the distinguishability matrix is ​​determined by the following formula: ; In the formula, similarity _ matrix is the cosine similarity matrix; clarity _ matrix is the boundary definition matrix; According to the preset distinguishability value of the distinguishability matrix, the flat image data set is reorganized in the form of a flat image classification task into the form of a tree image classification task; The calculating of the cosine similarity matrix based on the feature matrix comprises: Based on the feature matrix, extracting first image features of a first type of image; the first type of image is an image of any category in the flat image data set; Based on the first image feature, calculating an average value of the first image feature to obtain a first category center; Based on the feature matrix, extracting second image features of a second type of image; the second type of image is an image of any category in the flat image data set; Based on the second image feature, calculating an average value of the second image feature to obtain a second category center; According to the first category center and the second category center corresponding to all category images in the flattened image data set, a cosine similarity matrix is ​​obtained using a cosine similarity formula; The step of calculating a boundary definition matrix based on the feature matrix comprises: Based on the feature matrix, a first data feature and a second data feature are obtained according to the data label; the first data feature and the second data feature are data features corresponding to any two category images in the flat image data set; the data label is a classification text of each category image in the flat image data set; Using the first data feature and the second data feature to train a support vector machine model, and obtaining an average distance from a support vector to a decision boundary and the number of support vectors; A boundary sharpness matrix is ​​obtained based on the average distance of the support vector to the decision boundary and the number of support vectors.

2. The method of reorganizing a flat image classification task into a tree-shaped image classification task according to claim 1, characterized in that: The cosine similarity matrix is ​​determined by the following formula: ; In the formula, A k is a Category I centre; B k A Category II centre; i +1 represents the number of rows in the cosine similarity matrix; j +1 represents the number of columns in the cosine similarity matrix; 0≤ k ≤ n -1.

3. The method of reorganizing a flat image classification task into a tree-shaped image classification task according to claim 1, characterized in that: The boundary definition matrix is ​​determined by the following formula: ; Where SVM_d is the average distance from the support vector to the decision boundary; SVM_n is the number of support vectors; i +1 represents the number of rows in the boundary sharpness matrix; j +1 represents the number of columns in the boundary sharpness matrix.

4. The method of reorganizing a flat image classification task into a tree-shaped image classification task according to claim 1, characterized in that: The step of reorganizing the flat image dataset in the form of a flat image classification task into a tree image classification task according to the preset distinguishability value of the distinguishability matrix comprises: Define the preset value of distinguishability; Based on the distinguishability matrix, obtaining a first cluster and a second cluster in the distinguishability matrix; the distance between the first cluster and the second cluster is the smallest compared to other clusters in the distinguishability matrix; Determine whether the distance between the first cluster and the second cluster is less than or equal to the preset distinguishability value, and if so, merge the first cluster and the second cluster to generate an intermediate cluster; According to the intermediate clusters and the unmerged clusters, the flat image dataset is reorganized in the form of a flat image classification task into the form of a tree image classification task.

5. The method of reorganizing a flat image classification task into a tree-shaped image classification task according to claim 4, characterized in that: The step of reorganizing the flat image dataset in the form of a flat image classification task into a tree image classification task according to the intermediate clusters and the unmerged clusters includes: Calculating the distinguishability distance between the intermediate clusters; Based on the distinguishability distance, the distinguishability matrix is ​​updated, and the step of determining whether the distance between the first cluster and the second cluster is less than or equal to the preset distinguishability value is repeatedly performed until the distance between the first cluster and the second cluster in the distinguishability matrix is ​​greater than the preset distinguishability value, thereby generating a target cluster; According to the first cluster, the second cluster, the intermediate cluster, the target cluster, and the unmerged cluster, the flat image dataset is reorganized in the form of a flat image classification task into the form of a tree image classification task.

6. The method of reorganizing a flat image classification task into a tree-shaped image classification task according to claim 5, characterized in that: The distinguishability distance between the intermediate clusters is determined by the following formula: ; In the formula, C ij is the middle cluster; C i is the first cluster; C j For the second cluster; k 0 indicates other clusters that are not currently merged; ; 。 7. A system for reorganizing a flat image classification task into a tree-shaped image classification task, characterized in that: include: An extraction module, wherein the extraction module is configured to: extract a feature matrix of a flattened image data set using a pre-trained neural network; The flat image dataset includes: a number of images of different categories; A first calculation module, the first calculation module is configured to: calculate a cosine similarity matrix and a boundary clarity matrix based on the feature matrix; the cosine similarity matrix represents the pairwise similarities of feature vectors of all categories in the feature matrix; the boundary clarity matrix represents the degree of separation of feature vectors of different categories in the feature matrix in space; The second calculation module is configured to calculate a distinguishability matrix based on the cosine similarity matrix and the boundary definition matrix; the distinguishability matrix is ​​determined by the following formula: ; In the formula, similarity _ matrix is the cosine similarity matrix; clarity _ matrix is the boundary definition matrix; A reorganization module, wherein the reorganization module is configured to: reorganize the flat image data set in the form of a flat image classification task into a form of a tree image classification task according to a preset value of the distinguishability of the distinguishability matrix; The step of calculating a cosine similarity matrix based on the feature matrix comprises: Based on the feature matrix, extracting first image features of a first type of image; the first type of image is an image of any category in the flat image data set; Based on the first image feature, calculating an average value of the first image feature to obtain a first category center; Based on the feature matrix, extracting second image features of a second type of image; the second type of image is an image of any category in the flat image data set; Based on the second image feature, calculating an average value of the second image feature to obtain a second category center; According to the first category center and the second category center corresponding to all category images in the flattened image data set, a cosine similarity matrix is ​​obtained using a cosine similarity formula; The step of calculating a boundary definition matrix based on the feature matrix comprises: Based on the feature matrix, a first data feature and a second data feature are obtained according to the data label; the first data feature and the second data feature are data features corresponding to any two category images in the flat image data set; the data label is a classification text of each category image in the flat image data set; Using the first data feature and the second data feature to train a support vector machine model, and obtaining an average distance from a support vector to a decision boundary and the number of support vectors; A boundary sharpness matrix is ​​obtained based on the average distance of the support vector to the decision boundary and the number of support vectors.