Image classification method and device, storage medium and computer device

Abstract

The application discloses an image classification method and device, a storage medium and a computer device. The method is applied to the computer device and comprises the following steps: in a self-defined image classification mode of an electronic album, in response to a triggering operation of a user on an update control of an image corresponding to a to-be-managed image category, updating the image in the to-be-managed image category, calling a training interface, obtaining a first image classification model corresponding to the self-defined image mode through the training interface, the model being pre-trained, and performing transfer training on all images of the to-be-managed image category according to the first image classification model, so as to obtain a first image classification model after transfer training. The classification of the updated image can be matched by using the first image classification model after transfer training, the classification accuracy is improved, finally, the to-be-classified images of the electronic album are classified by using the trained first image classification model, the image classification efficiency and accuracy are improved, and the image searching efficiency is further improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an image classification method, apparatus, storage medium, and computer equipment. Background Technology

[0002] With the evolution and iteration of network communication technology, mobile phones, tablets and other portable computer devices, as carriers of modern communication technology, have gradually become indispensable communication tools in people's daily life and work.

[0003] Portable computers, as multifunctional communication tools, increasingly rely on their photo album functions as an important way to record work and life. Currently, photo album functions primarily categorize images by storage time or by application source. These sorting methods make it inconvenient for users to find specific types or images, and over time, the sheer number of images can make them impossible to find. Summary of the Invention

[0004] This application provides an image classification method, apparatus, storage medium, and computer device, which can improve image classification efficiency and further improve image search efficiency.

[0005] This application provides an image classification method, including:

[0006] In the custom image classification mode of the electronic photo album, in response to the user's trigger operation of the update control for the image corresponding to the image to be managed, the image in the image to be managed is updated, and the training interface is called;

[0007] The first image classification model corresponding to the custom image classification mode is obtained through the training interface. The first image classification model has been pre-trained.

[0008] The first image classification model is transferred to train based on all images of the image category to be managed, so as to obtain the first image classification model after transfer training.

[0009] The first image classification model trained by the transfer learning is used to classify the images to be classified in the electronic photo album.

[0010] This application also provides an image classification device, including:

[0011] The update module is used to update the images in the image category to be managed in the custom image classification mode of the electronic photo album in response to the user's trigger operation of the update control for the image corresponding to the image category to be managed;

[0012] The calling module is used to invoke the training interface based on the updated image;

[0013] The acquisition module is used to acquire the first image classification model corresponding to the custom image classification mode through the training interface. The first image classification model has been pre-trained.

[0014] The training module is used to perform transfer training on the first image classification model based on all images of the image category to be managed, so as to obtain the first image classification model after transfer training.

[0015] The classification module is used to classify the images to be classified in the electronic photo album using the first image classification model trained by the transfer learning.

[0016] This application also provides a computer-readable storage medium storing a plurality of instructions adapted to be loaded by a processor to execute any of the above-described image classification methods.

[0017] This application also provides a computer device, including a processor and a memory, wherein the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used for steps in the image classification method described above.

[0018] The image classification method, apparatus, storage medium, and computer equipment provided in this application set a custom image classification mode for an electronic photo album. In this custom image classification mode, in response to a user's triggering operation of the update control for the image corresponding to the image category to be managed, the images in the image category to be managed are updated, and a training interface is called. It is understood that the training interface is called only when the user triggers the update operation, thus triggering training. The first image classification model corresponding to the custom image mode is obtained through the training interface. This model has been pre-trained. Transfer training is then performed on all images of the image category to be managed based on the first image classification model to obtain the transfer-trained first image classification model. During training, only the pre-trained first image classification model needs to be transferred to all images of the image category to be managed, instead of training all images of all image categories again. This improves training speed and allows the transfer-trained first image classification model to match the classification situation after image updates, improving classification accuracy. Finally, the trained first image classification model is used to classify the images to be classified in the electronic photo album, thus improving image classification efficiency and accuracy, and further improving image search efficiency. Attached Figure Description

[0019] The technical solution and other beneficial effects of this application will become apparent from the following detailed description of specific embodiments in conjunction with the accompanying drawings.

[0020] Figure 1 A schematic diagram of the image classification switch for an electronic photo album provided in an embodiment of this application.

[0021] Figure 2 An example diagram of the image category display interface provided in the embodiments of this application.

[0022] Figure 3 This is a flowchart illustrating the image classification method provided in an embodiment of this application.

[0023] Figure 4 The image shown is a simplified diagram of the VGG-16 model training process.

[0024] Figure 5 This is a schematic diagram of a sub-process of the image classification method provided in the embodiments of this application.

[0025] Figure 6 This is a schematic diagram of another sub-process of the image classification method provided in the embodiments of this application.

[0026] Figure 7 This is another schematic diagram of the image classification method provided in the embodiments of this application.

[0027] Figure 8 This is a schematic diagram of the structure of the image classification device provided in the embodiments of this application.

[0028] Figure 9 A schematic diagram of the structure of a computer device provided in an embodiment of this application.

[0029] Figure 10 Another structural schematic diagram of the computer device provided in the embodiments of this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] This application provides an image classification method, apparatus, storage medium, and computer device. Any of the image classification apparatuses provided in this application can be integrated into a computer device, which may include a terminal or server. The terminal may include smartphones, tablets, wearable devices, robots, smart TVs, smart air conditioners, smart in-vehicle devices, personal computers (PCs), etc. The server may be an independent physical server, a service node in a blockchain system, a server cluster consisting of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, and big data and artificial intelligence platforms.

[0032] The image classification method in this application embodiment is applied to the electronic photo album in the computer device, which integrates electronic photo album or similar functions.

[0033] The electronic photo album of this application provides at least one image classification mode, such as a custom image classification mode. In the custom image classification mode, users can manage image categories, including adding image categories, deleting image categories, and setting names for image categories. In other embodiments, the electronic photo album provides at least two image classification modes, such as a custom image classification mode and a normal image classification mode. The normal image classification mode is the default image classification mode of the electronic photo album. In the normal image classification mode, there are fixed image categories, and users cannot add or delete image categories.

[0034] This embodiment uses an electronic photo album with both custom image classification modes and standard image classification modes as an example. After starting the computer device, the user can trigger the image classification switch control to open the electronic photo album, such as... Figure 1 As shown, after turning on the image classification switch, users can trigger either the default image classification control or the custom image classification control to select "Default Image Classification" or "Custom Image Classification," respectively. "Default Image Classification" and "Custom Image Classification" correspond to the normal image classification mode and the custom image classification mode, respectively.

[0035] The triggering operation for the corresponding control can be any of the following: touch operation, click operation, double-click operation, long press operation, right-click operation, etc. It can be set according to the actual scenario and there is no specific limitation.

[0036] When "Custom Image Category" is selected, all image categories under the custom image category mode will be displayed in the image category display interface, such as... Figure 2As shown, image categories include Image Category 1, Image Category 2, Image Category 3, and Image Category 4 (such as an unknown category). The image category display interface also allows users to add, delete, or modify image categories under the custom image classification mode.

[0037] The image classification method, apparatus, computer-readable storage medium, and computer device in the embodiments of this application will be described in detail below. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0038] Figure 3 This is a flowchart illustrating an image classification method provided in an embodiment of this application. The image classification method includes the following steps.

[0039] 101. In the custom image classification mode of the electronic photo album, in response to the user's trigger operation of the update control for the image corresponding to the image to be managed, the image in the image to be managed is updated, and the training interface is called.

[0040] When "Custom Image Classification" is selected in the digital photo album, the custom image classification mode is entered. In this mode, the image categories to be managed can be either newly added or existing categories. Existing categories are those from the pre-trained first image classification model; all other categories are newly added.

[0041] For example, in response to a triggering operation of the add control for adding a new image category in the custom image classification mode, the settings area for the new image category is displayed; in response to a triggering operation of the save control for saving the new image category in the settings area, the new image category entered in the settings area is saved, and the new image category is set as an image category to be managed.

[0042] like Figure 2 As shown, the "+" in the lower right corner of the image category display interface corresponds to the new control for adding an image category. Triggering this control displays the settings area for the new image category, where you can set the name of the new image category, etc. The settings area also includes a save control. Triggering the save control saves the new image category.

[0043] The update control for the image corresponding to the image category to be managed can be either a new image control for adding images within the image category (e.g., for a new image category, the new image control adds the image corresponding to that new image category), or for an existing image category, it adds the image corresponding to that existing image category. The update control for the image corresponding to the image category to be managed can also be a delete image control for deleting images within the image category to be managed; or it can be a move image control for moving an image from one image category to another. Correspondingly, the image categories to be managed include the image category corresponding to the image being moved and the image category corresponding to the image being moved. For example, if an image is moved from image category 2 to image category 3, then the image categories to be managed include image category 2 and image category 3.

[0044] The triggering operation can be any one of the following: a click operation, a double-click operation, a long-press operation, a right-click operation, etc., such as a click operation.

[0045] In the custom image classification mode, in response to the user's triggering operation of the update control for the image corresponding to the image category to be managed, the images in the image category to be managed are updated. For example, for a new image category in an electronic photo album, new images need to be added to the new image category so that the first image classification model corresponding to the custom image classification mode can learn the image features of the new image category.

[0046] It is important to note that the triggering operation in this step must be initiated by the user. User-initiated triggering means that the user updates the existing image category or adds a new image category so that the first image classification model can learn the image features actively updated by the user and improve the image classification accuracy.

[0047] Each time a user actively triggers an update to an image in a managed image category, a training interface is invoked. This training interface is used to call the first image classification model corresponding to the custom image classification mode for further transfer training of the first image classification model. This training interface is pre-encapsulated and is invoked when an update to an image in a managed image category is actively triggered.

[0048] In this embodiment, the update control for the image corresponding to the image category to be managed is taken as the new image control for the newly added image category. The following description of transfer training is also based on this case.

[0049] 102. Obtain the first image classification model corresponding to the custom image classification mode through the training interface. This first image classification model has been pre-trained.

[0050] The first image classification model corresponding to the custom image classification mode is pre-trained and stored in the first path of the computer device before being called by the training interface. The first image classification model corresponding to the custom image classification mode is obtained from the first path through the training interface.

[0051] The first image classification model can be a neural network model, such as a convolutional neural network model, a recurrent neural network model, or a long short-term memory network model. It can also be other models that achieve the same or similar functions. The Visual GeometryGroup Network (VGG-16) within a convolutional neural network will be used as an example for illustration.

[0052] Before training the VGG-16 model, it's necessary to define the image category names and the number of image categories in the electronic photo album, and then select the images corresponding to each category to complete the image classification for each category. Each image category has corresponding label data, which identifies the image category to which it belongs. This label data is encoded using one-hot encoding. The images corresponding to each category are divided into training set samples, validation set samples, and test set samples. The training set samples are used for training input, the validation set samples are used to test the classification performance of the VGG-16 model after each training round, and the test set samples are used to verify the classification performance of the model after training.

[0053] Next, the VGG-16 model was built. The VGG-16 model specifically includes 5 convolutional groups and 3 fully connected layers. The 5 convolutional groups have 2, 2, 3, 3, and 3 convolutional layers respectively. The convolutional stride, total number of samplings, initial values ​​of weights, activation function, backpropagation optimization algorithm, loss function, and preset number of iterations were set for each convolutional layer of the VGG-16 model.

[0054] In VGG-16, the convolutional layers can be depthwise separable convolutional layers to reduce the size of the network model and improve its running efficiency.

[0055] The backpropagation optimization algorithm used is the Adam algorithm. The Adam algorithm enables the model to converge quickly during training, and the adaptive learning rate decay strategy allows for continuous fine-tuning of weights even after overall model convergence, resulting in a higher fit to the image sample set. To achieve the best image classification results, it is necessary to experimentally determine the optimal core parameters for the Adam algorithm: learning rate and batch size.

[0056] The loss function used can be the cross-entropy loss function, because the VGG-16 model has a simple sequential structure and the backpropagation efficiency of gradients is limited. Using the cross-entropy loss function can better avoid the vanishing gradient situation and achieve better image classification results.

[0057] The training steps are as follows: Figure 4 As shown, the specific steps include the following.

[0058] 201. Obtain the training set samples. This training set samples include image samples corresponding to each image category.

[0059] 202. Classify the constructed VGG-16 model based on the training set samples to obtain the training image classification results of the training set samples.

[0060] Specifically, image samples corresponding to each image category can be sampled from the training set to obtain the current training set samples. These sampled training set samples are then used as input to the VGG-16 model for training, yielding the training image classification results. The training image classification results are also encoded using One-Hot encoding.

[0061] 203. Based on the classification results of the training images and the corresponding label data of the training set samples, the cross-entropy loss value is determined using the cross-entropy loss function.

[0062] Specifically, the cross-entropy loss value can be calculated using the cross-entropy loss function based on the One-Hot encoding value of the training image classification result of the training set samples and the One-Hot encoding value of the corresponding label data.

[0063] 204. Check if the cross-entropy loss value is greater than the preset loss value.

[0064] The preset loss value can be 10. -6 Alternatively, if the cross-entropy loss value is greater than the preset loss value, then proceed to step 205; otherwise, proceed to step 207.

[0065] 205. Check if the preset number of iterations has been reached.

[0066] If the preset number of iterations is reached, proceed to step 207; otherwise, proceed to step 206.

[0067] 206. Perform gradient backpropagation. During gradient backpropagation, the Adam algorithm is used for optimization, and the weight parameters are updated. Then proceed to step 202.

[0068] 207, stop training, and obtain the first image classification model after training.

[0069] This yields the trained VGG-16 model, which is then used as the first image classification model.

[0070] 103. Transfer training is performed on the first image classification model based on all images of the image category to be managed, so as to obtain the first image classification model after transfer training.

[0071] After obtaining the pre-trained first image classification model through the training interface, if the image category to be managed is a new image category and the update control is a new image control, the first image classification model is transferred to train based on all images of the image category to be managed and the custom configuration parameters. After transfer training, the first image classification model can complete the image classification of the subsequent new image category. If the image category to be managed is a new image category and the update control is a delete image control, the classification of the first image classification model is more accurate after transfer training.

[0072] If the image category to be managed is an existing image category, then regardless of whether the update control is to add or delete an image control, all images of all current image categories will be obtained, and the custom configuration parameters in the parameter configuration file of the first image classification model will be modified. Based on the custom configuration parameters and all images of all image categories, the first image classification model will be trained to obtain the trained first image classification model, so that the trained first image classification model can classify according to the user's needs.

[0073] In one embodiment, such as Figure 5 As shown, the steps of transferring the training of the first image classification model based on all images of the image category to be managed include the following steps 301 to 303.

[0074] 301. Obtain the parameter configuration file of the first image classification model, and modify the custom configuration parameters in the parameter configuration file according to the image category to be managed.

[0075] The parameter configuration file for the first image classification model in the custom image classification mode is modifiable. This configuration file includes custom configuration parameters such as image category name, number of image categories, preset iteration count, learning rate, and batch size. Modifying the custom configuration parameters in the parameter configuration file depends on the image category to be managed. For example, if the image category is newly added, the name of the new image category is added, and the number of image categories, preset iteration count, learning rate, and batch size are changed (e.g., decreasing the preset iteration count, learning rate, and batch size). If the image category to be managed is an existing image category, the preset iteration count, learning rate, and batch size are changed (e.g., decreasing the preset iteration count, learning rate, and batch size). When modifying custom configuration parameters for both newly added and existing image categories, the modified value for the same configuration parameter can be the same or different, but generally different.

[0076] 302. Obtain all images corresponding to the image category to be managed, and use all images as input images for the first image classification model, where each input image corresponds to an image category to be managed.

[0077] That is, obtain all images of the newly added image category and use all images as input images for the first image classification model.

[0078] 303. Based on the custom configuration parameters and the input image, perform transfer training on the first image classification model.

[0079] Since transfer training is designed for situations where a new image control has been added, it first modifies the fully connected layers of the first image classification model based on the number of image categories specified in the custom configuration parameters, ensuring that the number of output categories matches the number of newly added image categories. Then, it modifies the training parameters in the transfer training based on other configuration parameters. Finally, it trains the first image classification model using the modified model and the input image. This enables the first image classification model to classify the newly added image categories.

[0080] The specific transfer training process is the same as the training process described above, and will not be repeated here.

[0081] 104. The first image classification model trained by transfer is used to classify the images to be classified in the electronic photo album.

[0082] In one embodiment, such as Figure 6 As shown, step 104 includes steps 401 to 403.

[0083] 401. When there are images to be categorized in the electronic photo album, the first categorization interface is called based on the images to be categorized.

[0084] The images to be classified include new images captured by the camera, or new images saved / downloaded through other applications.

[0085] When an image to be categorized is detected in the electronic photo album, the first classification interface is called based on the image to be categorized. The first classification interface is used to call the first image classification model corresponding to the custom image classification mode, so as to further classify the image based on the first image classification model. The first classification interface is pre-encapsulated and is called when an image to be categorized exists in the electronic photo album.

[0086] 402, obtain the images to be classified in the electronic photo album and the first image classification model after transfer training through the first classification interface.

[0087] 403. The first image classification model trained by transfer is used to classify the image to be classified in order to determine the target image category of the image to be classified.

[0088] The image to be classified is input into the first image classification model after transfer training. The first image classification model is used to classify the image to obtain the same number of probability values ​​as the number of image categories. For example, if there are 10 image categories, 10 probability values ​​are obtained. The image category with the highest probability value is taken as the target image category, and the image to be classified is displayed in the target image category display area of ​​the electronic photo album.

[0089] Thus, the first image classification model trained by transfer learning is used to achieve automatic classification of the image to be classified.

[0090] Figure 7 This is another schematic flowchart of the image classification method provided in the embodiments of this application, which includes the following steps.

[0091] 501. Obtain a pre-trained image classification model. Based on the pre-trained image classification model, obtain a first image classification model and a second image classification model. Encapsulate the first image classification model and the second image classification model to obtain a first model file and a second model file.

[0092] The first image classification model and the second image classification model correspond to the custom image classification mode and the ordinary image classification mode, respectively. Both the first and second image classification models are derived from pre-trained image classification models. For example, a backup of the pre-trained image classification model can be made, and the parameter configuration file of one copy can be modified to obtain the first and second image classification models. The parameter configuration file in the first image classification model can be modified. The training steps for the image classification models are described above in the section on training the first image classification model, and will not be repeated here.

[0093] The parameter configuration file in the first image classification model can be changed, and the weight parameters in the first image classification model can be updated; the parameter configuration file in the second image classification model is fixed and cannot be changed, and the weight parameters in the second image classification model are fixed and cannot be changed.

[0094] The first image classification model and the second image classification model are encapsulated to obtain a first model file and a second model file, including: generating a corresponding first model file from the first image classification model according to a preset format, and generating a corresponding second model file from the second image classification model according to the same format. For example, generating a .h5 format model file.

[0095] 502. Save the first model file to the first path corresponding to the custom image classification mode, and save the second model file to the second path corresponding to the ordinary image classification mode.

[0096] The first and second paths are pre-defined, corresponding to the custom image classification mode and the normal image classification mode, respectively. The first model file is saved to the first path, and the second model file is saved to the second path.

[0097] 503. In the custom image classification mode of the electronic photo album, in response to the user's trigger operation of the update control for the image corresponding to the image to be managed, the image in the image to be managed is updated, and the training interface is called.

[0098] 504. Obtain the first model file corresponding to the custom image classification mode from the first path through the training interface. The first model file encapsulates the first image classification model corresponding to the custom image classification mode.

[0099] 505. Transfer training is performed on the first image classification model based on all images of the image category to be managed, so as to obtain the first image classification model after transfer training.

[0100] 506. When there are images to be categorized in the electronic photo album, the first categorization interface is called based on the images to be categorized.

[0101] 507, obtain the image to be classified in the electronic album through the first classification interface and call the first image classification model after transfer training from the first path.

[0102] 508. The first image classification model trained by transfer is used to classify the image to be classified in order to determine the target image category of the image to be classified.

[0103] 509. In the normal image classification mode of the electronic photo album, when there are images to be classified in the electronic photo album, the second classification interface is called according to the images to be classified.

[0104] When the electronic photo album is set to normal image classification mode, it means that the electronic photo album is in normal image classification mode. In normal image classification mode, when there is an image to be classified in the electronic photo album, the second classification interface is called according to the image to be classified. The second classification interface is used to call the second image classification model corresponding to the normal image classification mode, so as to further classify the image according to the second image classification model. The second classification interface is pre-encapsulated and is called when there is an image to be classified in the electronic photo album in normal image classification mode.

[0105] 510. Obtain the image to be classified in the electronic album through the second classification interface and call the second model file corresponding to the ordinary image classification mode from the second path. The second model file encapsulates the second image classification model corresponding to the ordinary image classification mode.

[0106] 511. The second image classification model is used to classify the image to be classified in order to determine the target image category of the image to be classified. The target image category is an image category that already exists in the second image classification model.

[0107] For any steps in this embodiment that are consistent with the steps described above but not described in detail, please refer to the description of the corresponding steps above, and they will not be repeated here.

[0108] This embodiment further describes the classification process when there are images to be classified in the electronic photo album under both the custom image classification mode and the normal image classification mode.

[0109] Based on the above embodiments, in the custom image classification mode of the electronic photo album, the image classification method further includes: displaying all image categories in the custom image classification mode in the image category display interface of the electronic photo album, including unknown categories; responding to the triggering operation of the delete control for deleting a certain image category on the image category display interface, classifying all images in the certain image category into the unknown category and deleting the certain image category; and modifying the custom configuration parameters in the parameter configuration file according to the deleted image category.

[0110] When the delete control for a specific image category is triggered on the image category display interface, such as when the delete control for image category 2 is triggered, all images corresponding to image category 2 are reclassified into the unknown category, and image category 2 is deleted. After deleting image category 2, the number of image categories parameter in the parameter configuration file is modified, and the fully connected layer is modified accordingly. It is important to note that transfer learning is not performed during deletion.

[0111] Furthermore, in the custom image classification mode of the electronic photo album, the image classification method also includes: determining the image corresponding to the image category to be managed from images of unknown categories.

[0112] In the above embodiments, it is understandable that when a corresponding image is added or deleted in a new image category, the training interface will be called and the transfer training mode will be entered. The classification effect will be continuously optimized by adjusting the model weight parameters to achieve the user's expected classification. When a corresponding image is added or deleted in an existing image category, the training interface will be called and the full training mode will be entered.

[0113] Based on the method described in the above embodiments, this embodiment will be further described from the perspective of an image classification device, which can be implemented as an independent entity or integrated into a computer device.

[0114] Please see Figure 8 , Figure 8 This application provides a specific description of an image classification apparatus, which is applied in a computer device. The image classification apparatus may include: an update module 601, a calling module 602, an acquisition module 603, a training module 604, and a classification module 605.

[0115] The update module 601 is used to update the images in the image category to be managed in the custom image classification mode of the electronic photo album in response to the user's trigger operation of the update control for the image corresponding to the image of the image category to be managed.

[0116] Module 602 is used to invoke the training interface based on the updated image.

[0117] The acquisition module 603 is used to acquire the first image classification model corresponding to the custom image classification mode through the training interface. The first image classification model has been pre-trained.

[0118] The training module 604 is used to perform transfer training on all images of the image category to be managed based on the first image classification model, so as to obtain the first image classification model after transfer training.

[0119] In one embodiment, the training module 604 is configured to obtain the parameter configuration file of the first image classification model and modify the custom configuration parameters in the parameter configuration file according to the image category to be managed; obtain all images corresponding to the image category to be managed and use all images as input images of the first image classification model, the input images corresponding to the image category to be managed; and perform transfer training on the first image classification model according to the custom configuration parameters and the input images.

[0120] The classification module 605 is used to classify the images to be classified in the electronic photo album using the first image classification model trained by the transfer learning.

[0121] In one embodiment, the classification module 605 includes a calling unit, an acquisition unit, and a classification processing unit. The calling unit is used to call a first classification interface based on the image to be classified when the electronic photo album contains an image to be classified. The acquisition unit is used to acquire the image to be classified and the first image classification model trained by transfer learning from the electronic photo album through the first classification interface. The classification processing unit is used to perform classification processing on the image to be classified using the first image classification model trained by transfer learning to determine the target image category of the image to be classified.

[0122] In one embodiment, such as Figure 8 As shown, the image classification device may further include a determining module 606. The determining module 606 is used to determine the image category to be managed. Specifically, in response to a trigger operation of the add control for adding an image category in the custom image classification mode, it displays a setting area for the new image category; in response to a trigger operation of the save control for saving the new image category in the setting area, it saves the new image category entered in the setting area and uses the new image category as the image category to be managed.

[0123] In one embodiment, such as Figure 8 As shown, the image classification device may further include a display module 607 and a deletion module 608. The display module 607 is used to display all image categories under the custom image classification mode in the image category display interface of the electronic photo album, including unknown categories. The deletion module 608 is used to, in response to a trigger operation of the deletion control on the image category display interface for deleting a certain image category, classify all images in the certain image category into the unknown category and delete the certain image category. Correspondingly, the determining module 606 is further used to determine the image corresponding to the image category to be managed from the images in the unknown category.

[0124] In one embodiment, the classification module 605 is further configured to classify the images to be classified in the electronic photo album using a second image classification model in a normal image classification mode, wherein the second image classification model has been pre-trained.

[0125] In one embodiment, the calling unit is further configured to, in the normal image classification mode of the electronic album, when there is an image to be classified in the electronic album, call the second classification interface according to the image to be classified; the obtaining unit is further configured to obtain the image to be classified in the electronic album and the second image classification model corresponding to the normal image classification mode through the second classification interface; the classification processing unit is further configured to use the second image classification model to perform classification processing on the image to be classified in order to determine the target image category of the image to be classified, wherein the target image category is an image category that already exists in the second image classification model.

[0126] In one embodiment, such as Figure 8 As shown, the image classification device may further include a preprocessing module 609, wherein the preprocessing module 609 is used to acquire a pre-trained image classification model, obtain a first image classification model and a second image classification model based on the image classification model; encapsulate the first image classification model and the second image classification model to obtain a first model file and a second model file; save the first model file to a first path corresponding to the custom image classification mode, and save the second model file to a second path corresponding to the ordinary image classification mode; correspondingly, the acquisition module 603 is used to acquire the first model file corresponding to the custom image classification mode from the first path, wherein the first model file encapsulates the first image classification model corresponding to the custom image classification mode; or acquire the second model file corresponding to the ordinary image classification mode from the second path, wherein the second model file encapsulates the second image classification model corresponding to the ordinary image classification mode.

[0127] In practice, the above modules can be implemented as independent entities or combined arbitrarily as the same or several entities. For the specific implementation of the above modules, please refer to the previous method embodiments. For the specific beneficial effects that can be achieved, please also refer to the beneficial effects in the previous method embodiments, which will not be repeated here.

[0128] In addition, embodiments of this application also provide a computer device, such as... Figure 9 As shown, the computer device 700 includes a processor 701 and a memory 702. At least one application program is installed in the computer device. The processor 701 and the memory 702 are electrically connected.

[0129] The processor 701 is the control center of the computer device 700. It connects various parts of the computer device through various interfaces and lines. By running or loading applications stored in the memory 702 and calling data stored in the memory 702, it performs various functions of the computer device and processes data, thereby monitoring the computer device as a whole.

[0130] In this embodiment, the processor 701 in the computer device 700 loads the instructions corresponding to the processes of one or more application programs into the memory 702 according to the following steps, and the processor 701 runs the application programs stored in the memory 702 to realize various functions, such as:

[0131] In the custom image classification mode of the electronic photo album, in response to the user's trigger operation of the update control for the image corresponding to the image category to be managed, the images in the image category to be managed are updated, and the training interface is called; the first image classification model corresponding to the custom image classification mode is obtained through the training interface, and the first image classification model has been pre-trained; the first image classification model is transferred and trained according to all images of the image category to be managed to obtain the transferred and trained first image classification model; the transferred and trained first image classification model is used to classify the images to be classified in the electronic photo album.

[0132] This computer device can implement the steps of any embodiment of the image classification method provided in this application. Therefore, it can achieve the beneficial effects that any image classification method provided in this invention can achieve, as detailed in the preceding embodiments, which will not be repeated here.

[0133] Figure 10 A specific structural block diagram of a computer device provided in an embodiment of the present invention is shown. This computer device can be used to implement the image classification method provided in the above embodiments. The computer device includes the following modules / units.

[0134] RF circuit 810 is used to receive and transmit electromagnetic waves, realizing the mutual conversion between electromagnetic waves and electrical signals, thereby enabling communication with communication networks or other devices. RF circuit 810 may include various existing circuit elements used to perform these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, Subscriber Identity Module (SIM) cards, and memory. RF circuit 810 can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices via wireless networks. The aforementioned wireless networks may include cellular telephone networks, wireless local area networks (WLANs), or metropolitan area networks (MANs). The aforementioned wireless networks may use various communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging, and short messages, and any other suitable communication protocols, including those that have not yet been developed.

[0135] The memory 820 can be used to store software programs (computer programs) and modules, such as the program instructions / modules corresponding to those in the above embodiments. The processor 880 executes various functional applications and data processing by running the software programs and modules stored in the memory 820. The memory 820 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 820 may further include memory remotely located relative to the processor 880, and these remote memories can be connected to the computer device 800 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0136] The input unit 830 can be used to receive input digital or character information, and to generate keyboard, mouse, joystick, optical, or trackball signal inputs related to user settings and function control. Specifically, the input unit 830 may include a touch-sensitive surface 831 and other input devices 832. The touch-sensitive surface 831, also known as a touch display screen (touchscreen) or touchpad, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch-sensitive surface 831), and drive the corresponding connection device according to a pre-set program. Optionally, the touch-sensitive surface 831 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 880, and can also receive and execute commands sent by the processor 880. In addition, the touch-sensitive surface 831 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 831, the input unit 830 may also include other input devices 832. Specifically, other input devices 832 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0137] Display unit 840 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces of computer device 800. These graphical user interfaces can be composed of graphics, text, icons, video, and any combination thereof. Display unit 840 may include display panel 841, optionally configured as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), or other similar forms. Further, touch-sensitive surface 831 may cover display panel 841. When touch-sensitive surface 831 detects a touch operation on or near it, it transmits the information to processor 880 to determine the type of touch event. Subsequently, processor 880 provides corresponding visual output on display panel 841 according to the type of touch event. Although in the figures, touch-sensitive surface 831 and display panel 841 are implemented as two separate components to achieve input and output functions, it is understood that touch-sensitive surface 831 and display panel 841 can be integrated to achieve input and output functions.

[0138] The computer device 800 may also include at least one sensor 850, such as a light sensor, an orientation sensor, a proximity sensor, and other sensors. As a type of motion sensor, a gravity acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometers, taps), etc. Other sensors that the computer device 800 may be equipped with, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0139] Audio circuitry 860, speaker 861, and microphone 862 provide an audio interface between the user and computer device 800. Audio circuitry 860 converts received audio data into electrical signals, which are then transmitted to speaker 861, where they are converted into sound signals for output. Conversely, microphone 862 collects sound signals, converts them into electrical signals, which are received by audio circuitry 860, converted back into audio data, and then processed by processor 880 before being transmitted via RF circuitry 810 to, for example, another computer device, or output to memory 820 for further processing. Audio circuitry 860 may also include an earphone jack to facilitate communication between peripheral headphones and computer device 800.

[0140] Computer device 800, through transmission module 870 (e.g., Wi-Fi module), can help users receive requests, send information, etc., providing users with wireless broadband internet access. Although transmission module 870 is shown in the figure, it is understood that it is not an essential component of computer device 800 and can be omitted as needed without changing the essence of the invention.

[0141] The processor 880 is the control center of the computer device 800. It connects to various parts of the mobile phone via various interfaces and lines. By running or executing software programs (computer programs) and / or modules stored in the memory 820, and by calling data stored in the memory 820, it performs various functions of the computer device 800 and processes data, thereby providing overall monitoring of the computer device. Optionally, the processor 880 may include one or more processing cores; in some embodiments, the processor 880 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 880.

[0142] The computer device 800 also includes a power supply 890 (such as a battery) that supplies power to various components. In some embodiments, the power supply may be logically connected to the processor 880 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 890 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0143] Although not shown, the computer device 800 also includes cameras (such as front-facing cameras and rear-facing cameras), Bluetooth modules, etc., which will not be described in detail here. Specifically, in this embodiment, the display unit of the computer device is a touch screen display, and the computer device also includes a memory and one or more programs (computer programs), wherein one or more programs are stored in the memory and configured to be executed by one or more processors. One or more programs contain instructions for performing the following operations:

[0144] In the custom image classification mode of the electronic photo album, in response to the user's trigger operation of the update control for the image corresponding to the image category to be managed, the images in the image category to be managed are updated, and the training interface is called; the first image classification model corresponding to the custom image classification mode is obtained through the training interface, and the first image classification model has been pre-trained; the first image classification model is transferred and trained according to all images of the image category to be managed to obtain the transferred and trained first image classification model; the transferred and trained first image classification model is used to classify the images to be classified in the electronic photo album.

[0145] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of the above modules, please refer to the previous method implementation examples, which will not be repeated here.

[0146] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructions (computer programs) or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. Therefore, embodiments of the present invention provide a storage medium storing multiple instructions that can be loaded by a processor to execute the steps of any embodiment of the image classification method provided by the present invention.

[0147] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0148] Since the instructions stored in the storage medium can execute the steps in any embodiment of the image classification method provided in the embodiments of the present invention, the beneficial effects that any image classification method provided in the embodiments of the present invention can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0149] The foregoing has provided a detailed description of an image classification method, apparatus, storage medium, and computer device provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image classification method, characterized in that, include: In the custom image classification mode of the electronic photo album, in response to the user's trigger operation of the update control for the image corresponding to the image to be managed, the image in the image to be managed is updated, and the training interface is called. The training interface is used to trigger the call to the first image classification model and to trigger the transfer training of the first image classification model. The first image classification model corresponding to the custom image classification mode is obtained through the training interface. The first image classification model has been pre-trained. The first image classification model is transferred to train based on all images of the image category to be managed, so as to obtain the first image classification model after transfer training, wherein the image category to be managed corresponds to the training labels of all images; The first image classification model trained by the transfer learning is used to classify the images to be classified in the electronic photo album.

2. The image classification method according to claim 1, characterized in that, The step of performing transfer training on the first image classification model based on all images of the image category to be managed includes: Obtain the parameter configuration file of the first image classification model, and modify the custom configuration parameters in the parameter configuration file according to the image category to be managed; Obtain all images corresponding to the image category to be managed, and use all images as input images for the first image classification model, wherein the input images correspond to the image category to be managed; The first image classification model is transferred to the custom configuration parameters and the input image.

3. The image classification method according to claim 1, characterized in that, The step of classifying the images to be classified in the electronic photo album using the first image classification model trained by transfer learning includes: When the electronic photo album contains images to be categorized, the first categorization interface is invoked based on the images to be categorized. The first classification interface is used to obtain the image to be classified in the electronic photo album and the first image classification model after transfer training. The first image classification model trained by the transfer learning is used to classify the image to be classified in order to determine the target image category of the image to be classified.

4. The image classification method according to claim 1, characterized in that, The category of the image to be managed is determined through the following steps: In response to the triggering operation of the new control for adding a new image category in the custom image classification mode, the setting area for the new image category is displayed; In response to a trigger operation on the save control in the settings area for saving new image categories, the new image category entered in the settings area is saved, and the new image category is set as an image category to be managed.

5. The image classification method according to claim 1, characterized in that, The image classification method further includes: The image category display interface of the electronic photo album displays all image categories under the custom image classification mode, including unknown categories; In response to a triggering operation of the delete control for deleting a certain image category on the image category display interface, all images in the certain image category are reclassified to the unknown category, and the certain image category is deleted; Determine the image corresponding to the category of the image to be managed from the images of the unknown category.

6. The image classification method according to claim 1, characterized in that, Also includes: In the normal image classification mode of the electronic photo album, when there are images to be classified in the electronic photo album, the second classification interface is called according to the images to be classified. The second classification interface is used to obtain the image to be classified in the electronic album and the second image classification model corresponding to the ordinary image classification mode. The second image classification model has been pre-trained. The image to be classified is classified using the second image classification model to determine the target image category of the image to be classified, wherein the target image category is an image category that already exists in the second image classification model.

7. The image classification method according to claim 1, characterized in that, The electronic photo album also includes a general image classification mode, which corresponds to a second image classification model. The second image classification model has been pre-trained. Before the step of obtaining the first image classification model corresponding to the custom image classification mode, the following steps are also included: Obtain a pre-trained image classification model, and derive a first image classification model and a second image classification model based on the image classification model; The first image classification model and the second image classification model are encapsulated to obtain a first model file and a second model file; Save the first model file to the first path corresponding to the custom image classification mode, and save the second model file to the second path corresponding to the ordinary image classification mode; The step of obtaining the first image classification model corresponding to the custom image classification mode includes: obtaining the first model file corresponding to the custom image classification mode from the first path, wherein the first model file encapsulates the first image classification model corresponding to the custom image classification mode.

8. An image classification device, characterized in that, include: The update module is used to update the images in the image category to be managed in the custom image classification mode of the electronic photo album in response to the user's trigger operation of the update control for the image corresponding to the image category to be managed; The calling module is used to call the training interface based on the updated image. The training interface is used to trigger the call to the first image classification model and to trigger transfer training on the first image classification model. The acquisition module is used to acquire the first image classification model corresponding to the custom image classification mode through the training interface. The first image classification model has been pre-trained. The training module is used to perform transfer training on the first image classification model based on all images of the image category to be managed, so as to obtain the first image classification model after transfer training, wherein the image category to be managed corresponds to the training labels of all images; The classification module is used to classify the images to be classified in the electronic photo album using the first image classification model trained by the transfer learning.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the image classification method according to any one of claims 1 to 7.

10. A computer device, characterized in that, The method includes a processor and a memory, the processor being electrically connected to the memory, the memory being used to store instructions and data, and the processor being used to execute the steps of the image classification method according to any one of claims 1 to 7.

Images