Image classification methods, electronic devices and storage media

By adjusting and optimizing the feature matrix and weight matrix of the image classification model, the problem of high computational cost of fully connected layers was solved, thereby improving the speed of image classification while maintaining accuracy.

CN117975079BActive Publication Date: 2026-11-17HON HAI PRECISION INDUSTRY CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211321457.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-10-26
Publication Date
2026-11-17
Estimated Expiration
2042-10-26

AI Technical Summary

Technical Problem

The massive computational demands of fully connected layers in existing image classification networks result in slow classification speeds, making it difficult to improve classification speed while maintaining accuracy.

Method used

By calculating the prediction accuracy of the first classification model, the target image is selected to adjust the model, generating a second classification model. The initial feature matrix is ​​then upgraded in dimensionality, the elements of the weight matrix are rearranged, and a target vector is generated for classification.

Benefits of technology

It reduces computational load, improves the speed and efficiency of image classification, avoids invalid computations, and ensures the accuracy of classification results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117975079B_ABST
    Figure CN117975079B_ABST
Patent Text Reader

Abstract

This application relates to image processing, providing an image classification method, electronic device, and storage medium. The method includes: acquiring a classification network, an image to be classified, a training image, and a test image; generating a first classification model based on the training image and the classification network; calculating the prediction accuracy of the first classification model on the test image; generating a second classification model based on the prediction accuracy, the test image, and the first classification model. The second classification model includes a flattening layer, a fully connected layer, and a classification layer. Based on the image to be classified and the flattening layer, an initial feature matrix is ​​obtained. If the dimension of the initial feature matrix is ​​less than the dimension of the initial weight matrix of the fully connected layer, the initial feature matrix is ​​increased in dimension to obtain a target feature matrix. The elements of the initial weight matrix are rearranged to obtain a target weight matrix. A target vector is generated based on the target weight matrix and the target feature matrix and input into the classification layer to obtain a classification result. This application can improve image classification efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to an image classification method, electronic device, and storage medium. Background Technology

[0002] In current image classification networks, the massive computational load in fully connected layers leads to long computation times, resulting in slow image classification speed. Therefore, how to improve the speed of image classification while ensuring classification accuracy has become a problem that needs to be solved. Summary of the Invention

[0003] In view of the above, it is necessary to provide an image classification method, electronic device and storage medium that can solve the problem of slow image classification speed caused by the difficulty in reducing the huge amount of computation in fully connected layers.

[0004] This application provides an image classification method, the method comprising: acquiring a classification network, acquiring an image to be classified, training images, and multiple test images; training the classification network based on the training images to obtain a first classification model; calculating the prediction accuracy of the first classification model on the multiple test images; if the prediction accuracy is less than a preset value, selecting a target image from the multiple test images to adjust the first classification model to obtain a second classification model, wherein the second classification model includes a flattening layer, a fully connected layer, and a classification layer; inputting the image to be classified into the second classification model and acquiring an initial feature matrix output from the flattening layer; if the dimension of the initial feature matrix is ​​less than the dimension of the initial weight matrix in the fully connected layer, performing dimensionality increase processing on the initial feature matrix to obtain a target feature matrix; rearranging the elements in the initial weight matrix to obtain a target weight matrix; generating a target vector based on the target weight matrix and the target feature matrix; and inputting the target vector into the classification layer to obtain the classification result of the image to be classified.

[0005] According to an optional embodiment of this application, training the classification network based on the training images to obtain a first classification model includes: calculating the loss value of the classification network based on the training images, adjusting the classification network based on the loss value until the loss value drops to its minimum and then stopping the adjustment, thereby obtaining the first classification model.

[0006] According to an optional embodiment of this application, calculating the loss value of the classification network based on the training images includes: the method for calculating the loss value is as follows:

[0007]

[0008]

[0009] Among them, L sup This refers to the aforementioned loss value. This refers to the loss value of the i-th training image, 2N refers to the total number of training images, i refers to the i-th training image in the training images, and y i The label category of the i-th training image refers to the labeled category of the i-th training image, and j refers to the j-th training image among the training images with the same labeled category as the i-th training image. j N refers to the labeled category of the j-th training image. yi This refers to the number of training images with the same labeled category as the i-th training image. i≠j This is the first indicator function, which takes the value zero if and only if i = j, and takes the value 1 if i ≠ j. The second indicator function is defined if and only if y i =y j When y is zero, i ≠y j Take 1 at time, || i≠k The third indicator function, z, takes zero if and only if i = k, and takes 1 if i ≠ k. i This refers to the unit vector z obtained by inputting the i-th training image into the classification network. j This refers to the unit vector obtained by inputting the j-th training image into the classification network, where k refers to any training image other than the i-th training image, and z k It refers to the unit vector obtained by inputting the k-th training image into the classification network, and τ is a preset scalar adjustment parameter.

[0010] According to an optional embodiment of this application, calculating the prediction accuracy of the first classification model for the multiple test images includes: obtaining the category labels of the multiple test images; inputting the multiple test images into the first classification model to obtain the prediction result for each test image; comparing the prediction result with the corresponding category label; determining the prediction result with the same category label as the target result; determining the test image corresponding to the target result as the target image; counting the first number of the multiple target images and the second number of the multiple test images; calculating the ratio of the first number to the second number; and obtaining the prediction accuracy.

[0011] According to an optional embodiment of this application, the step of selecting a target image from the plurality of test images to adjust the first classification model and obtain a second classification model includes: performing clustering processing on the plurality of target images to generate a plurality of cluster centers, inputting the target image corresponding to each cluster center into the first classification model for training, until the prediction accuracy is greater than or equal to the preset value, thereby obtaining the second classification model.

[0012] According to an optional embodiment of this application, if the dimension of the initial feature matrix is ​​smaller than the dimension of the initial weight matrix in the fully connected layer, then performing dimensionality-upgrading processing on the initial feature matrix to obtain the target feature matrix includes: counting the number of rows and columns of the initial feature matrix; multiplying the number of rows and columns to obtain a target product; performing prime factorization on the target product to obtain multiple prime factors; combining any two identical prime factors from the multiple prime factors into a prime factor pair; and calculating the product of the two prime factors in the prime factor pair to obtain the feature product. Each prime factor can only be combined once. A target prime factor pair is selected from the prime factor pairs based on the target product and the feature product. One prime factor from the target prime factor pair is extracted to obtain a feature prime factor. A feature value is generated based on the logarithm of the target prime factor pair and the feature prime factor. The target prime factor pair is replaced with zero among the multiple prime factors. After the replacement is completed, all non-zero prime factors are multiplied together to obtain the target value. Based on the configuration value, the target value, and the feature value, the initial feature matrix is ​​upgraded to obtain the target feature matrix.

[0013] According to an optional embodiment of this application, rearranging the elements of the initial weight matrix to obtain the target weight matrix includes: determining the last element of each column in the initial weight matrix as the target element, repeatedly adjusting the element at the beginning of each column to the end of each column, and moving the positions of the other elements in each column sequentially to the beginning of the column until the target element is moved to the beginning of the column and the rearrangement stops, thereby obtaining the target weight matrix.

[0014] According to an optional embodiment of this application, generating a target vector based on the target weight matrix and the target feature matrix includes: multiplying the target weight matrix and the target feature matrix to obtain the target vector.

[0015] This application provides an electronic device, the electronic device comprising:

[0016] Memory, storing at least one instruction; and

[0017] The processor executes the at least one instruction to implement the image classification method.

[0018] This application provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the image classification method.

[0019] As can be seen from the above technical solutions, this application calculates the prediction accuracy of the first classification model for the multiple test images, and then compares the prediction accuracy with the preset value to determine whether to adjust the first classification model. This can improve the prediction ability of the second classification model. Since only the target image is selected to adjust the first classification model during the adjustment process, the computational load during the adjustment process can be reduced, thereby improving the generation efficiency of the second classification model. The image to be classified is input into the second classification model, and the initial feature matrix output from the flattening layer is obtained. If the dimension of the initial feature matrix is ​​smaller than the dimension of the initial weight matrix in the fully connected layer, the initial feature matrix is ​​upgraded to obtain the target feature matrix. By changing the dimension of the target feature matrix, the number of parameters in each operation can be increased. The initial weight matrix of the fully connected layer is obtained, and the elements of the initial weight matrix are rearranged to obtain the target weight matrix. The target weight matrix and the target feature matrix are multiplied and accumulated to obtain the target vector. The target vector is then input into the classification layer to obtain the classification result of the image to be classified. Since the order of operations is changed after rearranging the elements in the initial weight matrix, it can be ensured that elements in the weight matrix participate in the operation in each operation, thereby avoiding blank output results (i.e., avoiding invalid operations). This reduces the number of operations and the time used for operation, thus speeding up the image classification process. Attached Figure Description

[0020] Figure 1 This is an application environment diagram of a preferred embodiment of the image classification method of this application.

[0021] Figure 2 This is a flowchart of a preferred embodiment of the image classification method of this application.

[0022] Figure 3 This is a schematic diagram of the structure of an electronic device that implements the image classification method of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] like Figure 1The diagram shown illustrates the application environment of a preferred embodiment of the image classification method of this application. The image classification method can be applied to one or more electronic devices 1, which communicate with a shooting device 2. The shooting device 2 can be a camera or other devices that perform the shooting action. For example, the shooting device 2 can capture images of target objects to obtain images to be classified. The target objects can be animals such as cats and dogs, or products such as water cups and toys.

[0025] The electronic device 1 is a device capable of automatically calculating parameter values ​​and / or processing information according to pre-set or stored instructions. Its hardware includes, but is not limited to: microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0026] The electronic device 1 can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0027] The electronic device 1 may further include network devices and / or user devices. The network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0028] The network where the electronic device 1 is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).

[0029] like Figure 2 The diagram shown is a flowchart of a preferred embodiment of an image classification method according to this application. Depending on different needs, the order of the steps in this flowchart can be adjusted according to actual detection requirements, and some steps can be omitted. The method is executed by an electronic device, such as... Figure 1 Electronic device 1 shown.

[0030] 101. Obtain the classification network, and obtain the image to be classified, the training image, and multiple test images.

[0031] In at least one embodiment of this application, the classification network refers to a network capable of predicting the category of the image to be classified.

[0032] In at least one embodiment of this application, the image to be classified refers to an image that needs to be classified.

[0033] In at least one embodiment of this application, the training images can be used to train the classification network.

[0034] In at least one embodiment of this application, the plurality of test images refers to images whose categories are known, and the plurality of test images can be used to calculate the accuracy of the classification network.

[0035] In at least one embodiment of this application, the electronic device constructs the classification network based on a convolutional neural network.

[0036] The convolutional neural network can be a VGG16 network, a ResNet network, or the like.

[0037] In at least one embodiment of this application, the electronic device acquires the image to be classified by:

[0038] The electronic device controls the imaging device to capture images of the objects to be classified, thereby obtaining the images to be classified.

[0039] The objects to be classified can be animals such as cats and dogs, or plants such as flowers.

[0040] In at least one embodiment of this application, the electronic device acquires the training images from a preset first database.

[0041] In at least one embodiment of this application, the electronic device obtains the plurality of test images and the category label of each test image from a preset second database. The category label refers to the true category of the test image. The second database may be a database such as CIFAR-10 or ImageNet.

[0042] In at least one embodiment of this application, in order to ensure that the brightness and size of the training image and the plurality of test images are more uniform, the acquired training image and the plurality of test images can be subjected to equalization and normalization processing.

[0043] The training images and the multiple test images refer to images of items that can include multiple categories such as animals (e.g., puppies, kittens, etc.) and plants (e.g., flowers, trees, etc.).

[0044] 102. The classification network is trained based on the training images to obtain a first classification model.

[0045] In at least one embodiment of this application, the first classification model refers to the model obtained by training the classification network using the training images.

[0046] In at least one embodiment of this application, the electronic device trains the classification network based on the training images to obtain a first classification model, including:

[0047] The electronic device calculates the loss value of the classification network based on the training images. Further, the electronic device adjusts the classification network based on the loss value until the loss value drops to the minimum and then stops adjusting, thus obtaining the first classification model.

[0048] By implementing the above methods, the classification network is trained based on the loss value until it converges, thus obtaining the first classification model, which can improve the classification accuracy of the first classification model.

[0049] Specifically, the electronic device calculates the loss value of the classification network based on the training images, including:

[0050] The electronic device inputs the training image into the classification network to obtain the labeled category of the training image, and the electronic device calculates the loss value based on the training image and the labeled category.

[0051] The method for calculating the loss value is as follows:

[0052]

[0053]

[0054] Among them, L sup This refers to the aforementioned loss value. This refers to the loss value of the i-th training image, 2N refers to the total number of training images, i refers to the i-th training image in the training images, and y i The label category of the i-th training image refers to the labeled category of the i-th training image, and j refers to the j-th training image among the training images with the same labeled category as the i-th training image. j N refers to the labeled category of the j-th training image. yi This refers to the number of training images with the same labeled category as the i-th training image. i≠j This is the first indicator function, which takes the value zero if and only if i = j, and takes the value 1 if i ≠ j. The second indicator function is defined if and only if y i =y j When y is zero, i ≠y j Take 1 at time, ||i≠k The third indicator function, z, takes zero if and only if i = k, and takes 1 if i ≠ k. i This refers to the unit vector z obtained by inputting the i-th training image into the classification network. j This refers to the unit vector obtained by inputting the j-th training image into the classification network, where k refers to any training image other than the i-th training image, and z k It refers to the unit vector obtained by inputting the k-th training image into the classification network, and τ is a preset scalar adjustment parameter.

[0055] 103. Calculate the prediction accuracy of the first classification model for the multiple test images.

[0056] In at least one embodiment of this application, the prediction accuracy refers to the probability that the first classification model correctly classifies the plurality of test images.

[0057] In at least one embodiment of this application, the electronic device calculates the prediction accuracy of the first classification model for the plurality of test images by including:

[0058] The electronic device acquires the category labels of the multiple test images, inputs the multiple test images into the first classification model, obtains the prediction result for each test image, compares the prediction result with the corresponding category label, and determines the prediction result with the same category label as the target result. Further, the electronic device determines the test image corresponding to the target result as the target image. Further still, the electronic device counts a first number of the multiple target images and a second number of the multiple test images. Further still, the electronic device calculates the ratio of the first number to the second number to obtain the prediction accuracy.

[0059] By using the above implementation method, by counting the first number of target images and the second number of multiple test images, and calculating the proportion of the first number in the second number, the prediction accuracy of the first classification model can be quickly calculated.

[0060] 104. If the prediction accuracy is less than a preset value, a target image is selected from the multiple test images to adjust the first classification model and obtain a second classification model, wherein the second classification model includes a flattening layer, a fully connected layer and a classification layer.

[0061] In at least one embodiment of this application, the target image refers to a portion of the test images corresponding to the first classification model when the prediction of the plurality of test images is accurate.

[0062] In at least one embodiment of this application, the electronic device selects a target image from the plurality of test images to adjust the first classification model to obtain a second classification model, including:

[0063] The electronic device performs clustering processing on multiple target images to generate multiple cluster centers. Further, the electronic device inputs the target image corresponding to each cluster center into the first classification model for training until the prediction accuracy is greater than or equal to the preset value, thereby obtaining the second classification model.

[0064] In this embodiment, the target images are clustered using the k-means clustering algorithm to generate multiple cluster centers.

[0065] The preset values ​​can be set by the user, and this application does not impose any restrictions.

[0066] Through the above implementation method, the target image is selected using a clustering algorithm to adjust the first classification model until the prediction accuracy of the first classification model is greater than the preset value, thereby obtaining the second classification model. This can improve the prediction accuracy of the second classification model. Since the target images are all correctly classified test images, adjusting the model using the target images can improve the classification ability of the second classification model.

[0067] In other embodiments of this application, if the cluster center contains any target image, then the target image is input into the first classification model for training; if the cluster center does not contain any target image, then the target image closest to the cluster center is input into the first classification model for training.

[0068] In other embodiments of this application, after adjusting the first classification model using the target image corresponding to each cluster center, if the adjusted first classification model cannot be greater than or equal to the preset value, the multiple target images need to be re-clustered to generate multiple new cluster centers for retraining the first classification model.

[0069] 105. Input the image to be classified into the second classification model and obtain the initial feature matrix output from the flattening layer.

[0070] In at least one embodiment of this application, the initial feature matrix refers to the matrix obtained after dimensionality reduction operation through the flattening layer.

[0071] In at least one embodiment of this application, the second classification model includes a convolutional layer, and the electronic device inputs the image to be classified into the second classification model and obtains the initial feature matrix output from the flattening layer, including:

[0072] The electronic device inputs the image to be classified into the convolutional layer for feature extraction to obtain a target matrix. The multiple dimensions of the target matrix are multiplied to obtain the result. The value 1 is used as the number of rows in the matrix, and the result is used as the number of columns in the matrix. Based on the number of rows and columns, the target matrix is ​​transformed into the initial feature matrix, which has a two-dimensional dimension.

[0073] The dimension of the target matrix is ​​greater than or equal to 3.

[0074] The process of generating the initial feature matrix is ​​basically the same as the process of inversely generating the target feature matrix described below, and will not be repeated here.

[0075] Through the above implementation method, the high-dimensional target matrix output from the convolutional layer is reduced in dimensionality to obtain the initial feature matrix. The dimensionality of the initial feature matrix can be controlled through the dimensionality reduction operation.

[0076] 106. If the dimension of the initial feature matrix is ​​smaller than the dimension of the initial weight matrix in the fully connected layer, the initial feature matrix is ​​upgraded to obtain the target feature matrix.

[0077] In at least one embodiment of this application, the target feature matrix refers to a matrix whose dimensions are the same as those of the initial weight matrix.

[0078] In at least one embodiment of this application, the electronic device performs dimensionality-upgrading processing on the initial feature matrix to obtain a target feature matrix, including:

[0079] The electronic device counts the number of rows and columns of the initial feature matrix. Further, the electronic device multiplies the number of rows and columns to obtain a target product. It then performs prime factorization on the target product to obtain multiple prime factors. Further still, the electronic device combines any two identical prime factors from the multiple prime factors into a prime factor pair and calculates the product of the two prime factors in the prime factor pair to obtain a feature product. Each prime factor can only be combined once. The electronic device selects a target prime factor pair from the prime factor pairs based on the target product and the feature product. Further, the electronic device extracts one prime factor from the target prime factor pair to obtain a feature prime factor. The electronic device generates a feature value based on the logarithm of the target prime factor pair and the feature prime factor. The electronic device replaces the target prime factor pair with zeros among the multiple prime factors. After the replacement, it multiplies all non-zero prime factors to obtain a target value. Based on the configuration value, the target value, and the feature value, the electronic device performs dimensionality-upgrading on the initial feature matrix to obtain the target feature matrix.

[0080] The target prime factor pair refers to a prime factor pair whose feature product is divisible by the target product, and the configuration value is 1.

[0081] It is understood that the target product is large enough and non-prime to ensure that at least two of the multiple prime factors are identical.

[0082] Specifically, the electronic device generates feature values ​​based on the logarithm of the target quality factor pair and the characteristic quality factor, including:

[0083] If the number of target quality factor pairs is a single pair, the electronic device determines the characteristic quality factor as the characteristic value. If the number of target quality factor pairs is multiple, the electronic device performs a multiplication operation on the characteristic quality factors to obtain the characteristic value.

[0084] Specifically, the electronic device performs dimensionality-upgrading processing on the initial feature matrix based on the configuration value, the target value, and the feature value to obtain the target feature matrix, which includes:

[0085] The electronic device uses the configuration value as the batch size, the target value as the number of channels, and the feature value as the number of rows and columns.

[0086] For example: the initial feature matrix has 1 row and 60 columns, that is: the initial feature matrix is

[0087] [1 1 1 1 1 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 1 1 1 1 1 2 2 2 22 3 3 3 3 3 1 1 1 1 1 1 1 1 1 1 1 3 3 3 3 3 2 2 2 2 2].

[0088] The configuration value is 1. The target value is calculated to be 15 using the above method. The feature value is 2. Based on the configuration value 1, the target value 15, and the feature value 2, the initial feature matrix is ​​upgraded to obtain the target feature matrix. The target feature matrix is:

[0089]

[0090] The target feature matrix contains a three-dimensional matrix, that is, the three-dimensional matrix includes 15 two-dimensional matrices, each of which has 2 rows and 2 columns.

[0091] In this embodiment, when the dimension of the initial weight matrix in the fully connected layer is four-dimensional, by uniformly converting the dimension of the initial feature matrix to four-dimensional, it can be ensured that the dimension of the target feature matrix input to the fully connected layer is consistent with the dimension of the initial weight matrix.

[0092] By implementing the above methods, the dimension of the initial feature matrix is ​​converted to be consistent with the dimension of the initial weight matrix, so that the target feature matrix can be directly multiplied with the initial weight matrix. This allows multiple two-dimensional matrices in the target feature matrix to be added to the operation. Since the number of parameters in each operation is increased, the operation speed of the fully connected layer can be improved.

[0093] 107. The elements in the initial weight matrix are rearranged to obtain the target weight matrix.

[0094] In at least one embodiment of this application, the target weight matrix refers to a matrix whose elements have been rearranged.

[0095] In at least one embodiment of this application, the electronic device rearranges the elements in the initial weight matrix to obtain a target weight matrix, including:

[0096] The electronic device determines the last element of each column in the initial weight matrix as the target element. Further, the electronic device repeatedly adjusts the element at the beginning of each column to the end of each column, and moves the other elements in each column sequentially to the beginning of the column until the target element is moved to the beginning of the column and the arrangement stops, thus obtaining the target weight matrix.

[0097] Through the above implementation method, all elements in the initial weight matrix can be rearranged, making the operation order of the elements in the initial weight matrix more conducive to fast multiplication with the target feature matrix.

[0098] 108. Generate a target vector based on the target weight matrix and the target feature matrix.

[0099] In at least one embodiment of this application, the target vector refers to the result of multiplying the target weight matrix and the target feature matrix.

[0100] In at least one embodiment of this application, the electronic device generates a target vector based on the target weight matrix and the target feature matrix, comprising:

[0101] The electronic device performs a multiplication operation between the target weight matrix and the target feature matrix to obtain the target vector.

[0102] By implementing the above methods, since the order of operations of the elements in the target weight matrix is ​​changed, it can be ensured that each element in the target feature matrix has a corresponding element in the target weight matrix to operate with during each operation, thereby improving the efficiency of the operation and reducing the operation time.

[0103] 109. Input the target vector into the classification layer to obtain the classification result of the image to be classified.

[0104] In at least one embodiment of this application, the classification result refers to the predicted category of the image to be classified by the second classification model. The classification result may include categories such as cat and dog.

[0105] In at least one embodiment of this application, the electronic device inputs the target vector into the classification layer to obtain the classification result of the image to be classified, including:

[0106] The electronic device inputs the target vector into the classification layer to obtain the probability of each category, and the electronic device takes the category corresponding to the highest probability value as the classification result.

[0107] The classification layer can be a softmax classifier.

[0108] By determining the category corresponding to the highest probability value as the classification result, the accuracy of the classification result can be improved.

[0109] As can be seen from the above technical solutions, this application calculates the prediction accuracy of the first classification model for the multiple test images, and then compares the prediction accuracy with the preset value to determine whether to adjust the first classification model. This can improve the prediction ability of the second classification model. Since only the target image is selected to adjust the first classification model during the adjustment process, the computational load during the adjustment process can be reduced, thereby improving the generation efficiency of the second classification model. The image to be classified is input into the second classification model, and the initial feature matrix output from the flattening layer is obtained. If the dimension of the initial feature matrix is ​​smaller than the dimension of the initial weight matrix in the fully connected layer, the initial feature matrix is ​​upgraded to obtain the target feature matrix. By changing the dimension of the target feature matrix, the number of parameters in each operation can be increased. The initial weight matrix of the fully connected layer is obtained, and the elements of the initial weight matrix are rearranged to obtain the target weight matrix. The target weight matrix and the target feature matrix are multiplied and accumulated to obtain the target vector. The target vector is then input into the classification layer to obtain the classification result of the image to be classified. Since the order of operations is changed after rearranging the elements in the initial weight matrix, it can be ensured that elements in the weight matrix participate in the operation in each operation, thereby avoiding blank output results (i.e., avoiding invalid operations). This reduces the number of operations and the time used for operation, thus speeding up the image classification process.

[0110] like Figure 3 The diagram shown is a schematic representation of the structure of an electronic device that implements the image classification method of this application.

[0111] In one embodiment of this application, the electronic device 1 includes, but is not limited to, a memory 12, a processor 13, and a computer program, such as an image classification program, stored in the memory 12 and executable on the processor 13.

[0112] Those skilled in the art will understand that the schematic diagram is merely an example of electronic device 1 and does not constitute a limitation on electronic device 1. It may include more or fewer components than shown in the diagram, or combine certain components, or different components. For example, electronic device 1 may also include input / output devices, network access devices, buses, etc.

[0113] The processor 13 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 13 is the computing core and control center of the electronic device 1, connecting various parts of the electronic device 1 through various interfaces and lines, and acquiring the operating system and installed applications and program code of the electronic device 1. For example, the processor 13 can acquire the images to be classified captured by the imaging device 2 through an interface.

[0114] The processor 13 acquires the operating system and various installed applications of the electronic device 1. The processor 13 acquires these applications to implement the steps in the various image classification method embodiments described above, for example... Figure 2 The steps are shown.

[0115] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and retrieved by the processor 13 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the process of retrieving the computer program from the electronic device 1.

[0116] The memory 12 can be used to store the computer programs and / or modules. The processor 13 implements various functions of the electronic device 1 by running or retrieving the computer programs and / or modules stored in the memory 12, and by calling the data stored in the memory 12. The memory 12 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 12 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0117] The memory 12 can be the external memory and / or internal memory of the electronic device 1. Furthermore, the memory 12 can be a physical memory, such as a memory module, a TF card (Trans-flash Card), etc.

[0118] If the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when the computer program is acquired by a processor, it can implement the steps of the various method embodiments described above.

[0119] The computer program includes computer program code, which may be in the form of source code, object code, accessible file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, and read-only memory (ROM).

[0120] Combination Figure 2 The memory 12 in the electronic device 1 stores multiple instructions to implement an image classification method. The processor 13 can acquire the multiple instructions to: acquire a classification network and acquire the image to be classified, training images, and multiple test images; train the classification network based on the training images to obtain a first classification model; calculate the prediction accuracy of the first classification model for the multiple test images; if the prediction accuracy is less than a preset value, select a target image from the multiple test images to adjust the first classification model to obtain a second classification model, wherein the second classification model includes a flattening layer, a fully connected layer, and a classification layer; input the image to be classified into the second classification model and acquire the initial feature matrix output from the flattening layer; if the dimension of the initial feature matrix is ​​less than the dimension of the initial weight matrix in the fully connected layer, perform dimensionality increase processing on the initial feature matrix to obtain a target feature matrix; rearrange the elements in the initial weight matrix to obtain a target weight matrix; generate a target vector based on the target weight matrix and the target feature matrix; input the target vector into the classification layer to obtain the classification result of the image to be classified.

[0121] Specifically, the processor 13's implementation method for the above instructions can be found in [reference needed]. Figure 2The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0122] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0123] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0124] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0125] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0126] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in this application may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. An image classification method, characterized in that, The image classification method includes: Obtain the classification network, and acquire the image to be classified, training images, and multiple test images; The classification network is trained based on the training images to obtain a first classification model; Calculating the prediction accuracy of the first classification model for the multiple test images includes: inputting the multiple test images into the first classification model to obtain the prediction result for each test image, comparing the prediction result of each test image with the corresponding category label, and determining the test image corresponding to the prediction result with the same category label as the target image; If the prediction accuracy is less than a preset value, the first classification model is adjusted based on multiple target images to obtain a second classification model, including: performing clustering processing on multiple target images to generate multiple cluster centers, inputting the target image corresponding to each cluster center into the first classification model for training, until the prediction accuracy is greater than or equal to the preset value, to obtain the second classification model, wherein the second classification model includes a flattening layer, a fully connected layer and a classification layer; The image to be classified is input into the second classification model, and the initial feature matrix output from the flattening layer is obtained; If the dimension of the initial feature matrix is ​​smaller than the dimension of the initial weight matrix in the fully connected layer, the initial feature matrix is ​​upgraded to obtain the target feature matrix. The elements in the initial weight matrix are rearranged to obtain the target weight matrix; Generate a target vector based on the target weight matrix and the target feature matrix; The target vector is input into the classification layer to obtain the classification result of the image to be classified.

2. The image classification method as described in claim 1, characterized in that, The step of training the classification network based on the training images to obtain the first classification model includes: The loss value of the classification network is calculated based on the training images; The classification network is adjusted based on the loss value until the loss value drops to its minimum, at which point the adjustment stops, and the first classification model is obtained.

3. The image classification method as described in claim 2, characterized in that, The calculation of the loss value of the classification network based on the training images includes: The method for calculating the loss value is as follows: in, This refers to the aforementioned loss value. It refers to the first The loss value of the training images, This refers to the total number of training images. It refers to the first training image. Zhang training images, It refers to the first The labeled categories of the training images This refers to the first... The training images with the same labeled category are the first training images. Zhang training images, It refers to the first The labeled categories of the training images It refers to and The number of training images with the same labeled category, It is the first indicator function if and only if When, take zero. Take 1 at time. It is the second indicator function if and only if When, take zero. Take 1 at time. It is a third indicator function if and only if When, take zero. Take 1 at time. This refers to The unit vector obtained by inputting it into the classification network. This refers to The unit vector obtained by inputting it into the classification network. It refers to except Any training image other than [the one mentioned above] This refers to The unit vector obtained by inputting it into the classification network. These are preset scalar adjustment parameters.

4. The image classification method as described in claim 1, characterized in that, The calculation of the prediction accuracy of the first classification model for the multiple test images further includes: Count the first number of multiple target images and the second number of multiple test images; The prediction accuracy is obtained by calculating the ratio of the first quantity to the second quantity.

5. The image classification method as described in claim 1, characterized in that, If the dimension of the initial feature matrix is ​​smaller than the dimension of the initial weight matrix in the fully connected layer, then the initial feature matrix is ​​subjected to dimensionality increase processing to obtain the target feature matrix, including: Count the number of rows and columns of the initial feature matrix; Multiply the number of rows of the matrix by the number of columns of the matrix to obtain the target product; The target product is decomposed into multiple prime factors. Combine any two identical prime factors from the plurality of prime factors into a prime factor pair, and calculate the product of the two prime factors in the prime factor pair to obtain the feature product. Each prime factor can only be combined once. Select a target prime factor pair from the prime factor pairs based on the target product and the feature product; Extract one quality factor from the target quality factor pair to obtain the characteristic quality factor; Feature values ​​are generated based on the logarithm of the target quality factor pair and the characteristic quality factor; Replace the target prime factor pair with zero among the plurality of prime factors, and after the replacement is completed, multiply all non-zero prime factors to obtain the target value; Based on the configuration value, the target value, and the feature value, the initial feature matrix is ​​subjected to dimensionality increase processing to obtain the target feature matrix.

6. The image classification method as described in claim 1, characterized in that, The step of rearranging the elements of the initial weight matrix to obtain the target weight matrix includes: The last element of each column in the initial weight matrix is ​​determined as the target element; Repeatedly move the element at the beginning of each column to the end of the column, and move the other elements in each column sequentially to the beginning of the column, until the target element is moved to the beginning of the column and the arrangement stops, thus obtaining the target weight matrix.

7. The image classification method as described in claim 1, characterized in that, The step of generating the target vector based on the target weight matrix and the target feature matrix includes: The target weight matrix is ​​multiplied by the target feature matrix to obtain the target vector.

8. An electronic device, characterized in that, The electronic device includes: Memory, storing at least one instruction; and The processor executes the at least one instruction to implement the image classification method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, which is executed by a processor in an electronic device to implement the image classification method as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Image multi-label classification method and device, electronic equipment and medium

    CN112465071A

  • Image classification method and device, equipment and storage medium

    CN113435522A