Image recognition method and system based on hierarchical category dynamic calculation

Abstract

The present invention relates to an image recognition method and system based on dynamic calculation of hierarchical categories, belonging to the field of computer image technology. The method comprises: S1: using a training set to train a small convolutional neural network to form a two-level image category hierarchy; S2: constructing a hierarchical neural network according to the hierarchy; S4: in a fine classification module, a path decision search is performed based on a coarse category to obtain an optimal path decision; S5: the output of the coarse classification module is used as a prediction of the coarse category and the path selection of the predicted coarse category to realize dynamic shunt calculation; S6: the prediction outputs of the coarse classification module and the fine classification module are fused, and the fusion result is used as the final category prediction; and the coarse and fine classification modules are jointly trained to improve the performance of the hierarchical network. The present invention utilizes the hierarchical structure of image categories to realize dynamic shunt calculation of categories in a hierarchical neural network, effectively improves the performance of the network, and helps to reduce the amount of calculation during reasoning.

Description

Technical Field

[0001] The invention belongs to the technical field of computer images, and relates to an image recognition method and system based on hierarchical category dynamic calculation. Background Art

[0002] With the rapid increase in the amount of labeled data and the improvement in computer hardware performance, the research on convolutional neural networks (CNNs) has rapidly emerged and has achieved outstanding results in image classification. The purpose of the convolutional layer is to learn the feature representation of the input, the activation function introduces nonlinearity into the network, and the pooling layer achieves translation invariance by reducing the resolution of the feature map. Many methods for improving the network structure have been proposed in existing research, but with the rapid increase in image categories, one type of problem that has emerged is the imbalanced visual similarity between categories, which makes some categories visually similar and usually difficult to distinguish.

[0003] Since the correlations between image categories vary, the current mainstream deep CNN uses fixed feedforward through-calculation and flat N-way classifiers, and mainstream research does not utilize the correlations between image categories. In existing research work on this issue, one method is to build a tree classifier to replace the traditional flat N-way classifier, and perform multi-granular classification from coarse to fine by adding hierarchical relationships in the labels. At each parent node, the classification process proceeds along the branch to the next fine-grained level until the last leaf node. For example, Kuang Z, Li Z, et al. proposed a hierarchical learning algorithm that supports large-scale plant species recognition in "Deep multi-task learning for large-scale image classification", using multi-task learning to jointly train related classifiers to improve recognition capabilities. Fan J, Zhao T, et al. used different layers of the neural network to extract multiple groups of multi-level deep features in HD-MTL: "Hierarchical deep multi-task learning for large-scale visual recognition", and learned the deep feature representation of the grouping through the similarity between classes. Zheng Y, Fan J et al. constructed a visual tree that is seamlessly integrated with multi-task learning to deal with data imbalance in "Discriminative Fast Hierarchical Learning for Multiclass Image Classification". In these research works, although the similarities between categories can be used to learn more discriminative tree classifiers, there is a problem that the classification errors of parent nodes will propagate from top to bottom along the tree structure, and the corresponding classifier needs to be trained for each node, which is not conducive to end-to-end training.

[0004] Another method is to use the category hierarchy information to construct a hierarchical network, embed the hierarchy directly into the network model, and form a multi-module hierarchical neural network. For example, Yan Z, Zhang H, et al. first proposed to build a hierarchical network consisting of a coarse classification module and multiple fine classification modules in "HD-CNN: hierarchical deep convolutional neural networks for large scale visual recognition"; Panchenko I, Khan A, et al. used the DBScan clustering method in "On expert-defined versus learned hierarchies for image classification" to make the category hierarchy contain only one coarse class and construct the corresponding fine classification module; Zhang H, Kuang Z, et al. realized large-scale plant species identification by integrating the outputs of multiple deep networks in "Aggregating diverse deep attention networks for large-scale plant species identification". However, in these research works, since the hierarchical neural network contains multiple fine classification modules and the number of fine classification modules increases linearly with the number of classification nodes, the overall computational and storage capacity of the hierarchical neural network increases significantly.

[0005] Therefore, a new neural network computing method is urgently needed to reduce the overall computational and storage requirements of existing hierarchical neural networks. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide an image recognition method and system based on dynamic calculation of hierarchical categories, so as to solve the problem of inter-level transmission of errors in tree-type classifiers in existing hierarchical learning methods, and the problem that multiple sub-classification modules in hierarchical networks lead to significant increase in calculation amount and storage amount. The present invention utilizes the hierarchical structure of image categories to realize dynamic shunt calculation of categories in hierarchical neural networks, effectively improves the performance of the network, and helps to reduce the amount of calculation during reasoning, which greatly promotes the development of related technical fields.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] 1. An image recognition method based on hierarchical category dynamic calculation, specifically comprising the following steps:

[0009] S1: Use the training set to train a small convolutional neural network, and calculate the confusion matrix and distance matrix. Use spectral clustering to cluster the distance matrix to obtain the coarse categories of the image, forming a two-level image category hierarchy from coarse to fine.

[0010] S2: construct a hierarchical neural network according to a two-level image category hierarchy; wherein the two-level image category hierarchy structures each correspond to a ResNet-style module, namely a coarse classification module and a fine classification module; (the coarse classification module performs the classification task of the coarse nodes, and the fine classification module performs the classification task of the fine nodes;)

[0011] S3: pre-training coarse classification module and fine classification module;

[0012] S4: In the fine classification module, a path decision search is performed based on the coarse category to obtain the optimal path decision corresponding to the coarse category;

[0013] S5: The output of the coarse classification module is used as the prediction of the coarse category and the path selection of the predicted coarse category, so as to realize the dynamic diversion calculation of images of different coarse categories in the fine classification module;

[0014] S6: The prediction outputs of the coarse classification module and the fine classification module are fused, and the fusion result is used as the final category prediction; and the coarse and fine classification modules are jointly trained to improve the performance of the hierarchical network.

[0015] Further, in step S1, a two-level image category hierarchy structure from coarse to fine is formed, which specifically includes the following steps:

[0016] S11: Train a small convolutional neural network using the training set of images;

[0017] S12: Randomly extract 10% of the images for each category from the training set and input them into the trained convolutional neural network to obtain a confusion matrix;

[0018] S13: A distance matrix is ​​calculated based on the obtained confusion matrix, and the distance matrix is ​​clustered using spectral clustering to obtain a category hierarchy. Fine categories in the same cluster belong to the same coarse category.

[0019] Further, in step S13, the distance matrix D is calculated according to the obtained confusion matrix A, and the calculation formula is:

[0020] D = 0.5 × ((EA) + (EA) T )

[0021] Where E is the identity matrix and T is the matrix transpose.

[0022] Further, step S4 specifically includes the following steps:

[0023] S41: Randomly extract 10% of the images for each category from the training set, and randomly generate binary path decisions for each coarse category. The corresponding "0" in the fine classification module indicates that the image of the coarse category crosses the residual block, and "1" indicates that the image of the coarse category uses the residual block;

[0024] S42: using the randomly extracted data set and the binary path decision, the corresponding classification accuracy is calculated in the fine classification module;

[0025] S43: sort the classification accuracy in descending order, select the top k groups of better path decisions; use the crossover and mutation operations in the genetic algorithm to generate new groups of path decisions based on the top k groups of better path decisions;

[0026] S44: Iteratively execute step S42 and step S43 until the optimal path decision is obtained.

[0027] Further, step S5 specifically includes the following steps:

[0028] S51: The image is input into a coarse classification module to predict the coarse category to which it belongs; according to the optimal path decision obtained in step S4, the path decision corresponding to the coarse category is obtained;

[0029] S52: The image input is classified into a fine classification module, and the residual block is selectively used according to the path decision to realize dynamic diversion calculation of different coarse categories.

[0030] Further, in step S52, the dynamic flow splitting calculation based on the coarse category is realized by the following formula, which is expressed as follows:

[0031]

[0032] Where: θ l is the input feature of the lth residual block, θ l+1 is the output feature of the lth residual block, ReLU is the nonlinear activation function, f(·) is the convolution operation, is the parameter weight of the lth residual block in the fine classification module, is the parameter bias of the lth residual block in the fine classification module, ρ∈{0,1} is the binary decision of the coarse category to the residual block. When ρ=0, the input feature θ l It is directly used as the input of the next residual block through a short connection, which means "skipping" the residual block. When ρ = 1, it is the normal calculation of the residual block, which means "using" the residual block.

[0033] Further, step S6 specifically includes the following specific steps:

[0034] S61: Fusing the prediction outputs of the coarse classification module and the fine classification module, and using the fused result as the final category prediction;

[0035] S62: Joint training: The loss function of joint training is based on the classification cross entropy function of the fusion output, and the cross entropy function of the coarse classification task is added as a subsidiary item. The coarse and fine modules are jointly trained to improve the classification accuracy of the hierarchical network.

[0036] Further, in step S62, the loss function of the joint training is:

[0037]

[0038] Where S(·) represents the cross entropy loss function, It represents the probability value of the fusion output of the coarse and fine classification module after normalization by the softmax function. represents the probability value of the output of the coarse classification module after normalization by the softmax function, y represents the fine category label, and y c represents the coarse category label, β represents the balance factor, γ represents the weight decay coefficient, and w represents the weight of the network parameter.

[0039] 2. A dynamic diversion computing system based on image category hierarchy, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the above-mentioned image recognition method based on hierarchical category dynamic computing when loaded into the processor.

[0040] The beneficial effects of the present invention are:

[0041] 1) Different from the traditional hierarchical neural network, the present invention adopts a separate sub-classification module to complete the sub-classification task, thus avoiding the problem that the number of traditional multiple sub-classification modules increases linearly with the number of nodes, resulting in a significant increase in the amount of calculation and storage.

[0042] 2) The present invention can realize the functions of multiple sub-classification modules in a traditional hierarchical network in only one sub-classification module. Through path decision search, the path decision is quickly obtained to realize the category diversion calculation.

[0043] 3) The present invention proposes a two-level feature learning scheme from coarse to fine, and constructs an image classification strategy that integrates coarse and fine hierarchical classifiers. Compared with the traditional tree-type SVM classifier, it is not necessary to train an SVM classifier for each node, which has more advantages in reducing computational cost and training cost.

[0044] 4) The present invention fuses the prediction outputs of the coarse and fine classification modules, and the final prediction result is jointly determined by the coarse and fine classification modules, avoiding the problem of error transmission between levels in traditional tree classifiers, and improving the classification performance of the network by potentially introducing category hierarchy information.

[0045] 5) The present invention implements dynamic diversion calculation of coarse categories in the fine classification module. During reasoning, some useless residual blocks are skipped, which reduces convolution operations and helps to accelerate reasoning.

[0046] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0048] Figure 1 is a flow chart of an image recognition method based on dynamic calculation of hierarchical categories of the present invention;

[0049] Figure 2 is an example diagram of path decision search according to the present invention;

[0050] Figure 3 This is an example diagram of dynamic flow splitting calculation according to the present invention;

[0051] Figure 4 It is a partial structural framework diagram of the dynamic flow distribution computing system based on the image category hierarchy of the present invention;

[0052] Figure 5 It is a hierarchical display diagram of image categories in the present invention;

[0053] Figure 6 It is a classification probability display diagram of some images in the present invention. DETAILED DESCRIPTION

[0054] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and the features in the embodiments can be combined with each other without conflict.

[0055] See also Figure 1 to Figure 6 , Figure 1 This is a flow chart of the image recognition method based on dynamic calculation of hierarchical categories in the present invention. The method is: using the image category hierarchy, integrating two basic network modules, namely the coarse classification module and the fine classification module, to form a hierarchical neural network. The coarse classification module predicts the coarse category to which each image belongs; the fine classification module further refines the fine categories contained in the coarse category. In order to extract more discriminative category features and avoid the problem of multiple fine classification modules in traditional hierarchical neural networks, each residual block in the fine classification module is selectively used according to the predicted coarse category to achieve different diversion calculations. In the fine classification module, each coarse category corresponds to the calculation path to which it belongs, Figure 2 For the example graph of path decision search, multiple binary path decisions for path decision are randomly generated for each coarse category, where "0" means skipping the residual block and "1" means using the residual block. The classification accuracy of the fine classification modules under different path decisions is sorted, and the optimal path decision is obtained by continuous iteration using crossover and mutation operations. Figure 3 This is an example diagram of dynamic diversion calculation implemented by the present invention. Each module is composed of multiple residual blocks. The coarse classification module outputs the coarse category prediction of the image and obtains the corresponding path selection based on the prediction result; the diversion calculation is performed in the fine classification module; the fusion output of the coarse and fine classification modules is used as the final result. Figure 5 It is an image category hierarchy display diagram implemented by the present invention; Figure 6 This is a diagram showing the classification probability of some images in the present invention. The present invention is aimed at the flow calculation of image categories, and mainly includes the following steps:

[0056] Step 1: Use the training set of images to train a small convolutional neural network. In order to utilize the hierarchical information of image categories, it is necessary to construct a category hierarchy from coarse to fine. By clustering the image features, n fine categories C = {C1, C2, ..., C n}Reassign to new m coarse categories For multiple fine categories in the same cluster, if their visual features are similar, they belong to the same coarse category, that is, a cluster is a coarse category. The present invention randomly extracts 10% of the images for each category from the training set, inputs them into the trained small convolutional neural network, and calculates the confusion matrix A between categories. The distance matrix D calculated by the confusion matrix is ​​calculated by the following formula:

[0057] D = 0.5 × ((EA) + (EA) T ) (1)

[0058] Where E is the identity matrix and T is the matrix transpose. In the distance matrix D, the element D ij The size of is used to measure the difficulty of distinguishing fine categories i and j. Using spectral clustering, n fine categories are clustered into m coarse categories, forming a many-to-one two-layer category hierarchy.

[0059] Step 2: Pre-train the coarse classification module and the fine classification module. According to the coarse and fine category labels of the image, the coarse classification module and the fine classification module are pre-trained independently. The cost of pre-training is relatively low, and only 40 epochs need to be trained at a large learning rate. The purpose of pre-training is: 1. Pre-improving the classification accuracy of the coarse classification module to reduce the misleading of subsequent path selection and output fusion; 2. Pre-improving the classification accuracy of the fine classification module is helpful for the implementation of path decision search.

[0060] Step 3: In the fine classification module, a path decision search is performed based on the coarse category to obtain the calculation path corresponding to the coarse category. The fine classification module consists of multiple residual blocks, and the short connection in each residual block can be directly used as the input and output of the two blocks before and after. Since the fine classification is located in the deep layer, specific category features are extracted. Each coarse category corresponds to a group of blocks with the most importance, and the corresponding calculation flow can be selected in the fine classification module. The residual part of the residual block has the largest amount of calculation. The image is directly input to the next block through the short connection, and the operation of the residual part is ignored, which effectively reduces the amount of calculation. Each coarse category can choose to cross or use the block to realize the diversion calculation. Multiple groups of binary codes are randomly generated to represent the corresponding path decisions, where "0" means "cross" and "1" means "use". Each group of decisions obtains the corresponding classification accuracy in the fine classification module. The top k best decisions are selected through the sorting accuracy, and the proportion of "1" can be set. Through the operations of crossover and mutation, the top k best decisions are used as the master to generate new decisions, and this cycle is repeated until the optimal path decision is obtained.

[0061] Step 4: Based on the optimal path decision found, the dynamic diversion calculation of the coarse category is performed in the fine classification module. Given an input image, the output of the coarse classification module is:

[0062] O c =ReLU(f(x;(w c ,b c ))) (2)

[0063] Where x∈R h×w×c is a three-dimensional image input; O c ∈R m is a one-dimensional feature output, where m is the number of coarse categories; f(·) is defined as a convolution operation; and ReLU is a nonlinear activation function. The coarse category prediction result of the image is argmax(O c ), the coarse category prediction result can correspond to the path selection of the image in the fine classification module. The fine classification module extracts fine category features. Given the input features, the normal output of the traditional l-th residual block is:

[0064]

[0065] Among them, θ l is the input feature of the lth residual block; θ l+1 is the output feature of the lth residual block. In the fine classification module, the diversion calculation is performed, and the residual block is "skipped" or "used", then formula (3) can be recalculated as:

[0066]

[0067] Among them, ρ∈{0,1} is the binary decision of the coarse category on the residual block. When ρ=0, the input feature θ l Directly use the short connection as the input of the next residual block, which means "skipping" the residual block; when ρ = 1, it is the normal calculation of the residual block, which means "using" the residual block. In the fine classification module, different coarse categories correspond to different path selections. Formula (4) can be used to realize dynamic diversion calculation based on coarse categories.

[0068] Step 5: The prediction outputs of the coarse classification module and the fine classification module are fused, and the fusion result is used as the final fine category prediction, which potentially introduces category hierarchy information into the network, thereby solving the problem of top-down inter-level propagation of errors in traditional tree classifiers. The image passes through the coarse classification module, and its output is: m is the number of coarse categories; according to the coarse category prediction argmax(O c ), and obtain the corresponding calculation path in the subdivision module. After the image is calculated by the subdivision module, its output is: n is the number of fine categories. The calculation of output fusion is:

[0069]

[0070] Among them, O v ∈R n is the prediction output after fusion; α represents the balance factor; G(i) represents the subscript of the coarse category to which the i-th fine category belongs. For example, if there are 5 classes in the fine category, they are assigned to two coarse categories {C′1, C′2}. If C′1={C1, C3, C4}, C′2={C2, C5}, then in the above example, G(3)=1, G(5)=2. According to formula (6), the fine category prediction output is linearly added to the corresponding coarse category prediction output. When the fine category label of an image is C1, and C1∈C′1. If only the fine classifier is used, the category prediction is wrong (the prediction result is C2), but because the difficulty of the coarse classification task is much lower than that of the fine classification task, the coarse category prediction of the image is correct (the prediction result is C′1). After the output of the coarse and fine classifiers is fused, the probability value of the fine category C2 will be suppressed and the probability value of the fine category C1 will be increased, which helps to improve the accuracy of the fine category prediction. The final fine category prediction of the image is argmax(O v ). The richness of hierarchical information has a certain relationship with the fusion effect. The more complex the hierarchical structure is, the more favorable it is for the fusion of coarse and fine classification decisions. However, the difficulty of the coarse classification task will also increase, resulting in errors in coarse category prediction and affecting the fusion result. Therefore, it is necessary to balance the difficulty of the coarse classification task and the fusion effect.

[0071] Step 6: Jointly train the coarse and fine classification modules to improve the performance of the hierarchical network. The prediction outputs of the coarse and fine classifiers are normalized by using the softmax function:

[0072]

[0073] Through training, the difference between the network output and the label is minimized, that is, the loss function is minimized. The loss function of the traditional image classification task is the cross entropy loss normalized by the softmax function. In the traditional classification task, the cross entropy loss function used is:

[0074]

[0075] Where x and y are the normalized predicted value and one-hot label, respectively. After output fusion, it is necessary to jointly train the coarse and fine classification modules in order to make them work together to improve the overall performance. An additional loss term is added to the loss function, namely the cross entropy loss of the coarse category. Under the l2 regularization term, the final loss function of the joint training is:

[0076]

[0077] in, Represents the predicted probability value of the image fine category, represents the predicted probability value of the coarse category of the image, and y represents the fine category label; c represents the coarse category label; β represents the balance factor; γ represents the weight decay coefficient; w represents the weight of the network parameter.

[0078] The hierarchical neural network of the present invention achieves excellent performance on real public datasets CIFAR-10, CIFAR-100 and SVHM. Figure 5 The image category hierarchy of CIFAR-10 and SVHN is shown, where each box represents a clustered coarse label and each image represents the original fine label. Compared with the original convolutional neural network, the hierarchical neural network can achieve similar classification accuracy with fewer FLOPs. For example, in the CIFDAR-10 dataset, the hierarchical neural networks built by ResNet-32, ResNet-56, and ResNet-110 have classification accuracies of 92.69%, 93.60%, and 93.79%, respectively, with FLOPs reduced by 29.42%, 39.08%, and 45.41%, respectively, all exceeding the accuracy of the original residual network. Figure 6 The top 5 probability values ​​of coarse and fine category predictions of different images in the test set of the CIFAR-100 dataset are displayed. When the original residual network misclassifies, the shunt calculation method in the hierarchical neural network of the present invention can obtain the correct classification result.

[0079] Figure 4 Shown is a partial framework diagram of a system related to the implementation of the present invention, the system includes: processors 410, 460, wireless network modules 441, 442, memories 430, 480, arithmetic unit 420, input unit 490, output unit 450, sensor 470 and other components. Figure 4 The architecture shown in the figure does not constitute a limitation of the system, and may include more or fewer modules than shown in the figure. Figure 4 A detailed introduction to each module of the system:

[0080] Memories 430 and 480 are used to store software programs, modules, and necessary data. Specifically, the processor 410 implements various functional applications and service provision by running software programs and modules stored in the memory 430. Due to the high similarity, the following introduction does not make special distinctions unless necessary. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program, etc.; the data storage area may store specific data: the memory 430 may store a trained network model weight file and an image cache sent by the terminal; the memory 480 may store image data obtained from the sensor 470, and classification results obtained from the wireless network module 442, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk device, a flash memory device, or other non-volatile solid-state storage device.

[0081] The wireless network modules 441 and 442 are the network communication modules of the remote server and the near terminal, respectively, and are responsible for remote interaction of requests and classification results between the server and the terminal. Specifically, the image requested for detection is sent to the wireless network module 441 through the wireless network module 442, and the classification result is sent through the wireless network module 441 after the classification is completed. It can be understood that in order to realize the image data transmission from the wireless network module 442 to the wireless network module 441, the transmission rate or bandwidth of the network needs to meet certain conditions, which have different definitions in different application scenarios.

[0082] The operation unit 420 is a computing device that carries the core process in the present invention, which is used to process the input image data and return the classification prediction result. The computing unit can be mainly composed of a data storage area 422 and an operation area 421, wherein the data storage area 422 is responsible for caching the data streams transmitted from the processor 410 and the memory 430, including the model weights of the hierarchical neural network, the optimal path decision code corresponding to the shunt calculation, and the image data to be classified; the operation area 421 is responsible for interacting with the internal data storage area 422 to directly process the calculation flow. For those skilled in the art, it can be understood that the operation unit 420 can adopt but is not limited to currently available high-performance parallel computing devices, such as independent devices encapsulated with high-performance computing chips, or distributed device clusters, and should have different forms in different application scenarios.

[0083] The sensor 470, the output unit 450 and the input unit 490 are components of the proximal terminal. The sensor 470 is responsible for collecting image data in a natural environment. The output unit 450 and the input unit 490 can be used to display information input by the user or information provided to the user. Specifically, the sensor 470 can be, but is not limited to, an independent camera (Webcam), a camera device integrated in a smartphone, or other available optical sensing devices. It should be noted that the image data collected by the sensor 470 needs to be resized by the processor 460 to meet the input size requirements of the hierarchical neural network. The input unit 490 is used to receive user input information, including a keyboard 491 and other input devices 492, such as a touch panel, a mouse, etc. The output unit 450 is used to display the user's interaction, including displaying the user's terminal input and displaying classification requests, etc. The display can be, but is not limited to, a liquid crystal display (LCD), an organic light emitting diode (OLED), etc. Figure 4 In the embodiment, the input unit 491 and the output unit 450 exist as independent modules, but can be designed in different forms according to the application scenarios. In some embodiments, the input unit 491 and the output unit 450 can be integrated into one I / O unit.

[0084] Processors 410 and 460 are respectively the logic processing units of the remote server and the near-end terminal, wherein the remote server refers to the machine deployed with the present invention, and the terminal includes but is not limited to general electronic devices such as smart phones and personal computers. Processor 410 is the scheduling unit of the present invention on the deployment device, responsible for calling and executing the modules or software programs stored in the memory 430, and interacting with the computing unit 420 and the wireless network module 441 for data and instructions. Processor 460 is the logic processing unit of the terminal device that initiates the request, responsible for processing the user's request, obtaining image data from the sensor 470, interacting with the wireless network module 442 and the memory 480, and outputting the results to the output unit 450. Due to the high similarity, the following introduction does not make a special distinction. The processor may include one or more processing units, and the processor may integrate an application processor and a modem, wherein the application processor is mainly responsible for processing such as operating systems and applications, and the modem is responsible for processing wireless communications. The above-mentioned modem is not a necessary module and may not be integrated into the processor.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. An image recognition method based on hierarchical category dynamic calculation, characterized in that: The method specifically comprises the following steps: S1: Use the training set to train a small convolutional neural network, and calculate the confusion matrix and distance matrix. Use spectral clustering to cluster the distance matrix to obtain the coarse categories of the image, forming a two-level image category hierarchy from coarse to fine. S2: construct a hierarchical neural network according to a two-level image category hierarchy; wherein the two-level image category hierarchy structures each correspond to a ResNet-style network module, namely a coarse classification module and a fine classification module; S3: pre-training coarse classification module and fine classification module; S4: In the fine classification module, a path decision search is performed based on the coarse category to obtain the optimal path decision corresponding to the coarse category; S5: The output of the coarse classification module is used as the prediction of the coarse category and the path selection of the predicted coarse category to realize the dynamic flow division calculation of images of different coarse categories in the fine classification module, which specifically includes the following steps: S51: The image is input into a coarse classification module to predict the coarse category to which it belongs; according to the optimal path decision obtained in step S4, the path decision corresponding to the coarse category is obtained; S52: The image input fine classification module selectively uses the residual block according to the path decision to achieve dynamic flow separation calculation of different coarse categories; S6: The prediction outputs of the coarse classification module and the fine classification module are fused, and the fusion result is used as the final category prediction; and the coarse and fine classification modules are jointly trained to improve the performance of the hierarchical network.

2. The image recognition method based on hierarchical category dynamic calculation according to claim 1 is characterized in that: In step S1, a two-level image category hierarchy is formed from coarse to fine, which specifically includes the following steps: S11: Train a small convolutional neural network using the training set of images; S12: Randomly extract 10% of the images for each category from the training set and input them into the trained convolutional neural network to obtain a confusion matrix; S13: A distance matrix is ​​calculated based on the obtained confusion matrix, and the distance matrix is ​​clustered using spectral clustering to obtain a category hierarchy. Fine categories in the same cluster belong to the same coarse category.

3. The image recognition method based on hierarchical category dynamic calculation according to claim 2 is characterized in that: In step S13, the distance matrix D is calculated based on the obtained confusion matrix A, and the calculation formula is: D=0.5×((EA)+(EA) T ) Where E is the identity matrix and T is the matrix transpose.

4. The image recognition method based on hierarchical category dynamic calculation according to claim 1 is characterized in that: Step S4 specifically includes the following steps: S41: Randomly extract 10% of the images for each category from the training set, and randomly generate binary path decisions for each coarse category. The corresponding "0" in the fine classification module indicates that the image of the coarse category crosses the residual block, and "1" indicates that the image of the coarse category uses the residual block; S42: using the randomly extracted data set and the binary path decision, the corresponding classification accuracy is calculated in the fine classification module; S43: sort the classification accuracy in descending order, select the top k groups of better path decisions; use the crossover and mutation operations in the genetic algorithm to generate new groups of path decisions based on the top k groups of better path decisions; S44: Iteratively execute step S42 and step S43 until the optimal path decision is obtained.

5. The image recognition method based on hierarchical category dynamic calculation according to claim 1 is characterized in that: In step S52, the dynamic flow splitting calculation based on the coarse category is implemented by the following formula, which is expressed as follows: Where: θ l is the input feature of the lth residual block, θ l+1 is the output feature of the lth residual block, ReLU is the nonlinear activation function, f(·) is the convolution operation, is the parameter weight of the lth residual block in the fine classification module, is the parameter bias of the lth residual block in the fine classification module, ρ∈{0,1} is the binary decision of the coarse category on the residual block; when ρ=0, the input feature θ l It is directly used as the input of the next residual block through a short connection, which means "skipping" the residual block; when ρ = 1, it is the normal calculation of the residual block, which means "using" the residual block.

6. The image recognition method based on hierarchical category dynamic calculation according to claim 1 is characterized in that: Step S6 specifically includes the following specific steps: S61: Fusing the prediction outputs of the coarse classification module and the fine classification module, and using the fused result as the final category prediction; S62: The loss function of joint training is a cross entropy function based on the fusion output. By adding the coarse classification cross entropy loss function as an auxiliary term, the coarse and fine modules are jointly trained to improve the classification accuracy of the hierarchical network.

7. The image recognition method based on hierarchical category dynamic calculation according to claim 6 is characterized in that: In step S62, the loss function of the joint training is: Where S(·) represents the cross entropy loss function, It represents the probability value of the fusion output of the coarse and fine classification module after normalization by the softmax function. represents the probability value of the output of the coarse classification module after normalization by the softmax function, y represents the fine category label, and y c represents the coarse category label, β represents the balance factor, γ represents the weight decay coefficient, and w represents the weight of the network parameter.

8. An image recognition system based on hierarchical category dynamic calculation, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into a processor, the image recognition method based on hierarchical category dynamic calculation according to any one of claims 1 to 7 is implemented.

Images