Image classification method and device, computer equipment and storage medium

Abstract

The invention relates to an image classification method and device, computer equipment and a storage medium. The method comprises the steps of obtaining a to-be-recognized image in a target task scene, and extracting image features from the to-be-recognized image through a plurality of pre-training deep learning networks in an image classification model; selecting target image features required in the target task scene from the image features; and performing classification identification on the target image features through a linear classifier in the image classification model to obtain a target category corresponding to the to-be-identified image. According to the scheme, for different image classification tasks, the target image features suitable for the image classification tasks are adaptively selected from the extracted image features, and image classification is performed based on the adaptively selected target image features, so that interference of redundant image features on image classification is avoided, and the accuracy of image classification is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image classification method, apparatus, computer equipment and storage medium. Background Art

[0002] Image classification technology has been widely used in many fields, such as medical image analysis, security monitoring, autonomous driving, industrial inspection, etc. Since the accuracy of image classification results is crucial in various fields, how to improve the accuracy of image classification results has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] Based on this, it is necessary to provide an image classification method, apparatus, computer device, computer readable storage medium and computer program product to improve the accuracy of image classification in response to the above technical problems.

[0004] In a first aspect, the present application provides an image classification method, which is applied to an embedded device, wherein an image classification model composed of multiple pre-trained deep learning networks and linear classifiers is deployed in the embedded device, and the training process of the image classification model in a target task scenario is completed in the embedded device, and the method includes:

[0005] Obtain the image to be identified in the target task scenario, and extract image features from the image to be identified through multiple pre-trained deep learning networks;

[0006] From the image features, select the target image features required in the target task scenario;

[0007] The target image features are classified and identified through a linear classifier to obtain the target category corresponding to the image to be identified.

[0008] In a second aspect, the present application provides an image classification device, which is configured in an embedded device, in which an image classification model composed of multiple pre-trained deep learning networks and linear classifiers is deployed, and the training process of the image classification model in the target task scenario is completed in the embedded device, and the device includes:

[0009] The extraction module is used to obtain the image to be identified in the target task scenario and extract image features from the image to be identified through multiple pre-trained deep learning networks;

[0010] The adaptive module is used to select the target image features required in the target task scenario from the image features;

[0011] The classification module is used to classify and identify the target image features through a linear classifier to obtain the target category corresponding to the image to be identified.

[0012] In a third aspect, the present application provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method when executing the computer program.

[0013] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0014] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and the computer program implements the steps in the above method when executed by a processor.

[0015] The above-mentioned image classification method, apparatus, computer equipment, computer-readable storage medium and computer program product, for different target tasks (i.e., image classification tasks), adaptively select target image features suitable for image classification tasks from the extracted image features, and perform image classification based on the adaptively selected target image features, thereby avoiding redundant image features from interfering with image classification and improving the accuracy of image classification. In addition, since the image classification model is composed of a pre-trained deep learning network and a linear classifier, the linear classifier only needs to be trained during the training of the image classification model, and the entire training process does not require too many computing resources, so that training for target classification task scenarios can be quickly implemented in embedded devices, and since the deep learning network in the image classification model is pre-trained, when training the model for the target task scenario, it is not necessary to collect and label a large amount of sample data for the target classification task in advance. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 An application environment diagram of an image classification method provided in an embodiment of the present application;

[0017] Figure 2 A flowchart of an image classification method provided in an embodiment of the present application;

[0018] Figure 3 A structural block diagram of an image classification device provided in an embodiment of the present application;

[0019] Figure 4 An internal structure diagram of a computer device provided in an embodiment of the present application;

[0020] Figure 5 An internal structure diagram of another computer device provided in an embodiment of the present application;

[0021] Figure 6 An internal structure diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0023] The image classification method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through a communication network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices, portable wearable devices and various embedded devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, image acquisition devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0024] like Figure 2 As shown, the embodiment of the present application provides an image classification method, which is applied to Figure 1The terminal device 102 (such as an embedded device) in is used as an example for explanation. In order to enable the embedded device to complete the image classification task intelligently, a deep learning model for image classification can be deployed in the embedded device, and the deployment process is as follows: for each specific classification task, a large amount of data needs to be manually collected and labeled, and then the deep network classification model is trained on the host side or the cloud, and then the trained deep network classification model is deployed to the embedded device. This deployment method has certain shortcomings: it needs to rely on the host side and the cloud device to complete the training of the deep network classification model to implement a specific classification task; for each specific classification task, a large amount of data needs to be re-collected and labeled. Based on this, the present application proposes a new image classification model, which is composed of multiple pre-trained deep learning networks and linear classifiers, wherein the pre-trained deep learning network can be obtained after training with large-scale data on the host side or the cloud, and only needs to be trained once for a specific classification task during training; after the image classification model is deployed to the embedded device, for a specific target task scenario (which can be a scenario for performing any specific classification task), a small number of training samples specific to the target task scenario are used in the embedded device to train the linear classifier in the image classification model, so that the training of the image classification model for the target task scenario can be completed in the embedded device. It can be understood that the reason for using a small number of training samples specific to the target task scenario for training is that the deep learning network used by the image classification model to extract image features is pre-trained, and does not need to be trained using the specific training samples in the target task scenario. After the image classification model for the target task scenario is trained in the embedded device, the image classification method of the present application can be implemented based on the image classification model. Specifically, the image classification method of the present application includes the following steps:

[0025] S201, obtaining an image to be identified in a target task scenario, and extracting image features from the image to be identified through multiple pre-trained deep learning networks.

[0026] In an embodiment of the present application, the target task scenario may be a specific image classification task scenario. For example, the target task scenario may be a scenario of image classification of various parts to identify different types of parts and classify and arrange them. Alternatively, the target task scenario may be a scenario of identifying the types of defects (such as scratches, cracks, etc.) in products based on images of products on a production line.

[0027] For the target task scenario, the image to be identified in the target task scenario can be obtained from the image acquisition device in the scenario, and the image to be identified can be input into the image classification model on the embedded device for classification and identification. It can be understood that before the image to be identified is input into the image classification model, the image to be identified can be preprocessed, wherein the preprocessing can include at least one of the following: 1. Image size adjustment to ensure that the adjusted image to be identified meets the size requirements of the image classification model for the input image; 2. Image normalization to make the pixel values ​​of the image distributed within a specific range; 3. Image denoising to remove noise in the image to be identified and improve image quality.

[0028] After the image to be identified is input into the image classification model on the embedded device, multiple pre-trained deep learning networks in the image classification model extract features from the input image to be identified to obtain multiple sets of image features; wherein the image features can be feature vectors, feature matrices or tensors, which are specifically determined by the structure and output mode of the network. It should be noted that the multiple pre-trained deep learning networks set in the image classification model can extract features from the image to be identified concurrently, so that enough generalized image features can be obtained.

[0029] For example, the image to be identified is x i , the m groups of features extracted by multiple pre-trained deep learning networks can be expressed as: Among them, f i 1 represents the first set of image features extracted from the image to be identified, ..., f i m Represents the mth group of image features extracted from the image to be identified.

[0030] S202: Select target image features required in a target task scenario from the image features.

[0031] In the present application, since different image classification tasks have different requirements for the selection of image features, in order to avoid interference of redundant image features in image classification, it is necessary to adaptively screen the numerous image features extracted in step S201 to select target image features suitable for the target task scenario.

[0032] In an optional implementation, the extracted multiple sets of image features can be screened according to the importance of each set of image features to the classification results in the target task scenario to obtain the required target image features. The importance of each set of image features to the classification results in the target task scenario is determined when the image classification model is trained in the target task scenario.

[0033] S203: Classify and identify the target image features through a linear classifier to obtain the target category corresponding to the image to be identified.

[0034] In the solution of the present application, the selected target image features can be input into the linear classifier of the image classification model, so that the linear classifier performs classification and recognition according to the target image features to obtain the target category of the image to be recognized.

[0035] In the present application scheme, for different target tasks (i.e., image classification tasks), target image features suitable for image classification tasks are adaptively selected from the extracted image features, and image classification is performed based on the adaptively selected target image features, thereby avoiding interference of extracted image features that are not suitable for image classification tasks on image classification results, thereby improving the accuracy of image classification. In addition, since the image classification model is composed of a pre-trained deep learning network and a linear classifier, the linear classifier only needs to be trained during the training of the image classification model, and the entire training process does not require too many computing resources, so that training for target classification task scenarios can be quickly implemented in embedded devices, and since the deep learning network in the image classification model is pre-trained, when training the model for the target task scenario, there is no need to collect and label a large amount of sample data for the target task scenario in advance.

[0036] In some embodiments, target image features required in a target task scenario are selected from the image features, including:

[0037] Obtain a feature application diversity set associated with the target task scenario; wherein the feature application diversity set includes a feature application score corresponding to each image feature, and the feature application score is used to measure the importance of the image feature in the target task scenario;

[0038] According to the feature application points corresponding to each image feature, the target image feature is selected from the image features.

[0039] In the present application scheme, the feature application subset associated with the target task scenario is determined when the image classification model is trained in the scenario, and the feature application scores associated with different task scenarios are different. It is understandable that since the feature application score is used to measure the importance of the image feature to the image classification result in the target task scenario, the larger the feature application score corresponding to a certain image feature, the more important this image feature is to the image classification result in the target task scenario, and therefore this image feature needs to be used as the target image feature.

[0040] For example, the image to be identified is x i , the m groups of features extracted by multiple pre-trained deep learning networks can be expressed as: The associated feature application diversity set in the target task scenario is {s1, s2, ... sm}. It can be seen that the first set of image features f i 1 The corresponding feature application scores are s1, ..., and the feature application scores corresponding to the mth group of image features are sm. In this way, it is only necessary to compare the size of the feature application scores corresponding to each group of image features to determine whether the image feature can be used as the target image feature. Optionally, the image features with the top N feature application scores can be used as the target image features, where N is a value greater than or equal to 1.

[0041] It can be understood that, through the above process, the target image features required by the target task scene can be adaptively selected, and image features that are not suitable for the target task scene can be prevented from interfering with subsequent image classification.

[0042] In some embodiments, the method further comprises:

[0043] In the process of training the image classification model under the target task scenario, determine the average distance between two training samples of different categories under the target feature group, the average distance between two training samples of the same category under the target feature group, and the mean of the classification confidence of each training sample output by the linear classifier; wherein the target feature group refers to any one of the multiple groups of image features extracted from the training samples by multiple pre-trained deep learning networks;

[0044] The feature application score corresponding to the target feature group is determined based on the average distance between pairwise training samples of different categories under the target feature group, the average distance between pairwise training samples of the same category under the target feature group, and the mean of the classification confidence of each training sample output by the linear classifier.

[0045] In the present application, the average distance between two training samples of different categories under the target feature group reflects the degree of difference between training samples of different categories under the target feature group. This average distance can be defined as the inter-class difference score. If the inter-class difference score is large, it means that the training samples of different categories are very different under the target feature group, and the feature is more effective in distinguishing different categories.

[0046] The average distance between two training samples of the same category under the target feature group reflects the closeness or similarity of the training samples of the same category under the target feature group. This average distance can be defined as the intra-class difference score. If the intra-class difference score is small, it means that the training samples of the same category are relatively similar under the target feature group, and the features have little difference in distinguishing samples within the category.

[0047] Since different classification tasks have different requirements for features, some tasks may need to focus more on features that distinguish different categories, while some tasks may require features that are more stable within the same category. Based on this, the feature application time of the target feature group is determined, and the average distance between the two training samples of different categories under the target feature group and the average distance between the two training samples of the same category under the target feature group are considered. In addition, the mean of the classification confidence of each training sample output by the linear classifier can reflect the contribution of the image features selected during the model training process to the accuracy of the classification results. Therefore, the confidence mean can also be considered when determining the feature application time of the target feature group.

[0048] It can be understood that according to the above steps, the feature application score corresponding to each feature group can be calculated, and when the feature application score corresponding to each feature group is saved through a set, the feature application subset set associated with the target task scene can be obtained. For example, the feature application subset set associated with the target task scene is {s1, s2, ... sm}. By determining the feature application score of the image feature group through information in three dimensions, the accuracy of the determined feature application score can be guaranteed.

[0049] In some embodiments, determining the average distance between two training samples of different categories under the target feature group includes:

[0050] According to the category of each training sample, pairs of training samples with different categories are grouped into sample pairs, and the number of sample pairs is determined;

[0051] If the target feature group is determined to be the jth group of image features extracted from the training samples by multiple pre-trained deep learning networks, then for each sample pair, the following operations are performed: extracting the jth group of image features from the two training samples included in the sample pair respectively; determining the distance between the jth group of image features of the two training samples;

[0052] According to the number of sample pairs and the distance between the jth group of image features of two training samples in each sample pair, the average distance between two training samples of different categories when the target feature group is the jth group of image features is determined.

[0053] Specifically, the average distance So between two training samples of different categories under the target image feature can be determined according to the following formula: j :

[0054]

[0055] Among them, y p represents the category of training sample p, y p represents the category of the training sample q, represents the jth group of image features extracted from training sample p, represents the jth group of image features extracted from training sample q, express and The Euclidean distance between them, K represents the number of sample pairs with different categories.

[0056] The above formula can be used to quickly and accurately calculate the average distance between two training samples of different categories under the target feature group.

[0057] In some embodiments, determining the average distance between two training samples of the same category under the target feature group includes:

[0058] According to the category of each training sample, two training samples of the same category are grouped into sample pairs, and the number of sample pairs is determined;

[0059] If the target feature group is determined to be the jth group of image features extracted from the training samples by multiple pre-trained deep learning networks, then for each sample pair, the following operations are performed: extracting the jth group of image features from the two training samples included in the sample pair respectively; determining the distance between the jth group of image features of the two training samples;

[0060] According to the number of sample pairs and the distance between the jth group of image features of two training samples in each sample pair, the average distance between the two training samples of the same category when the target feature group is the jth group of image features is determined.

[0061] Specifically, the average distance Si between two training samples of the same category under the target feature group can be determined according to the following formula: j :

[0062]

[0063] Among them, y p represents the category of training sample p, y p represents the category of the training sample q, represents the jth group of image features extracted from training sample p, represents the jth group of image features extracted from training sample q, express and The Euclidean distance between them, K represents the number of sample pairs with the same category.

[0064] The above formula can be used to quickly and accurately calculate the average distance between two training samples of the same category under the target feature group.

[0065] In some embodiments, the feature application score corresponding to the target feature group is determined according to the average distance between two training samples of different categories under the target feature group, the average distance between two training samples of the same category under the target feature group, and the mean of the classification confidence of each training sample output by the linear classifier, including:

[0066] Obtain the weight coefficients corresponding to the average distances of pairwise training samples of different categories under the target feature group, the average distances of pairwise training samples of the same category under the target feature group, and the mean of the classification confidence of each training sample output by the linear classifier;

[0067] According to the weight coefficient, the average distance between pairwise training samples of different categories under the target feature group, the average distance between pairwise training samples of the same category under the target feature group, and the classification confidence of each training sample output by the linear classifier are weighted and summed, and the weighted sum result is used as the feature application score corresponding to the target feature group.

[0068] Specifically, the feature application score S corresponding to the target feature group can be determined according to the following formula: j :

[0069] S j =aSo j +bSi j +cSp j ;

[0070] Among them, Sp j is the mean of the classification confidence of each training sample output by the linear classifier; So j is the average distance between two training samples of different categories under the target feature group, Si j is the average distance between two training samples of the same category under the target feature group; a, b, c are the preset weight coefficients respectively.

[0071] It is understandable that when determining the feature application score of the target feature group, taking into account the different emphasis on these factors in different tasks, different weight coefficients are configured for the average distance between pairwise training samples of different categories in the target feature group, the average distance between pairwise training samples of the same category in the target feature group, and the mean of the classification confidence of each training sample output by the linear classifier, so as to balance the importance of different factors and make the feature application score of the target feature group more in line with the current target task scenario.

[0072] In some embodiments, the multiple pre-trained deep learning networks include an ImageNet classification network, an image edge extraction network, and an image texture classification network; and the linear classifier is a logistic regression classifier.

[0073] Among them, the ImageNet classification network is a deep learning network trained based on the ImageNet classification dataset, which is a 1000-category classification network; the image edge extraction network is a deep learning network trained based on the image edge extraction dataset; the image texture classification network is a deep learning network trained based on the image texture classification dataset.

[0074] In the present application, the network skeletons of the ImageNet classification network, the image edge extraction network, and the image texture classification network can all select ResNet-50, and use multiple intermediate layers of the network skeleton as feature extraction layers. For example, "layer2.3.relu", "layer3.5.relu", and "layer4.2.relu" of the network skeleton are selected as feature extraction layers at three different levels.

[0075] On this basis, multiple pre-trained deep learning networks are used to extract image features from the image to be identified, including:

[0076] Extract image features from the image to be identified through the three feature extraction layers of the ImageNet classification network;

[0077] Image features are extracted from the image to be identified through three feature extraction layers of the image edge extraction network;

[0078] Image features are extracted from the image to be identified through the three feature extraction layers of the image texture classification network.

[0079] In this way, multiple groups of image features can be extracted through three pre-trained deep learning networks.

[0080] In the present application scheme, by setting up three pre-trained deep learning networks, as many generalized image features as possible can be extracted from the image to be identified for subsequent screening, which can avoid poor image classification results due to failure to extract key features; the linear classifier is a logistic regression classifier. Since the logistic regression classifier is a lightweight classifier, it does not need to occupy too many embedded computing resources during the model training process, thereby ensuring the training efficiency of the model.

[0081] It should be understood that, although the steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0082] Based on the same inventive concept, the embodiment of the present application also provides an image classification device. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more image classification device embodiments provided below can refer to the limitations on the image classification method above, and will not be repeated here.

[0083] like Figure 3 As shown, an embodiment of the present application provides an image classification device 300, which is configured in an embedded device, in which an image classification model composed of multiple pre-trained deep learning networks and linear classifiers is deployed, and the training process of the image classification model in the target task scenario is completed in the embedded device. The device includes:

[0084] An extraction module 301 is used to obtain an image to be identified in a target task scenario, and extract image features from the image to be identified through multiple pre-trained deep learning networks;

[0085] The adaptive module 302 is used to select the target image features required in the target task scenario from the image features;

[0086] The classification module 303 is used to classify and identify the target image features through a linear classifier to obtain the target category corresponding to the image to be identified.

[0087] In some embodiments, in selecting target image features required in a target task scenario from image features, the adaptive module 302 is specifically used to:

[0088] Obtain a feature application diversity set associated with the target task scenario; wherein the feature application diversity set includes a feature application score corresponding to each image feature, and the feature application score is used to measure the importance of the image feature in the target task scenario;

[0089] A target image feature is selected from at least one image feature according to the feature application score corresponding to each image feature.

[0090] In some embodiments, the apparatus further comprises:

[0091] The first determination module is used to determine the average distance between two training samples of different categories under the target feature group, the average distance between two training samples of the same category under the target feature group, and the mean of the classification confidence of each training sample output by the linear classifier during the process of training the image classification model under the target task scenario; wherein the target feature group refers to any one of multiple groups of image features extracted from the training samples by multiple pre-trained deep learning networks;

[0092] The second determination module is used to determine the feature application score corresponding to the target feature group based on the average distance between two training samples of different categories under the target feature group, the average distance between two training samples of the same category under the target feature group, and the mean of the classification confidence of each training sample output by the linear classifier.

[0093] In some embodiments, in determining the average distance between two training samples of different categories under the target feature group, the first determination module includes:

[0094] A first sample processing unit is used to group two pairs of training samples of different categories into sample pairs according to the category of each training sample, and determine the number of sample pairs;

[0095] A first distance calculation unit is used for, if it is determined that the target feature group is the jth group of image features extracted from the training samples by multiple pre-trained deep learning networks, then for each sample pair, performing the following operations: extracting the jth group of image features from two training samples included in the sample pair respectively; and determining the distance between the jth group of image features of the two training samples;

[0096] The second distance calculation unit is used to determine the average distance between two training samples of different categories when the target feature group is the jth group of image features according to the number of sample pairs and the distance between the jth group of image features of two training samples in each sample pair.

[0097] In some embodiments, in determining the average distance between two training samples of the same category under the target feature group, the first determination module further includes:

[0098] A second sample processing unit is used to group two training samples of the same category into sample pairs according to the category of each training sample, and determine the number of sample pairs;

[0099] A third distance calculation unit is used for, if it is determined that the target feature group is the jth group of image features extracted from the training samples by the multiple pre-trained deep learning networks, then for each sample pair, performing the following operations: extracting the jth group of image features from the two training samples included in the sample pair respectively; and determining the distance between the jth group of image features of the two training samples;

[0100] The fourth distance calculation unit is used to determine the average distance between two training samples of the same category when the target feature group is the jth group of image features according to the number of sample pairs and the distance between the jth group of image features of two training samples in each sample pair.

[0101] In some embodiments, in determining the feature application points corresponding to the target feature group according to the average distances of pairwise training samples of different categories under the target feature group, the average distances of pairwise training samples of the same category under the target feature group, and the mean of the classification confidences of the training samples output by the linear classifier, the second determination module is specifically used to:

[0102] Obtain the weight coefficients corresponding to the average distances of pairwise training samples of different categories under the target feature group, the average distances of pairwise training samples of the same category under the target feature group, and the mean of the classification confidence of each training sample output by the linear classifier;

[0103] According to the weight coefficient, the average distance between pairwise training samples of different categories under the target feature group, the average distance between pairwise training samples of the same category under the target feature group, and the classification confidence of each training sample output by the linear classifier are weighted and summed, and the weighted sum result is used as the feature application score corresponding to the target feature group.

[0104] In some embodiments, the multiple pre-trained deep learning networks of the image classification model include an ImageNet classification network, an image edge extraction network, and an image texture classification network; and the linear classifier is a logistic regression classifier.

[0105] Each module in the above-mentioned image classification device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0106] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data related to the implementation of the image classification method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above-mentioned image classification method are implemented.

[0107] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, the steps in the above-mentioned image classification method are implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0108] Those skilled in the art will understand that Figure 4 or Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0109] In some embodiments, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0110] In some embodiments, Figure 6 The figure shows an internal structure diagram of a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0111] In some embodiments, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0112] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0113] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0114] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image classification method, characterized in that: Applied to an embedded device, wherein an image classification model composed of a plurality of pre-trained deep learning networks and a linear classifier is deployed in the embedded device, and the training process of the image classification model in a target task scenario is completed in the embedded device, the method comprising: Acquire an image to be identified in a target task scenario, and extract image features from the image to be identified through the multiple pre-trained deep learning networks; Selecting target image features required in the target task scenario from the image features; The target image features are classified and identified by the linear classifier to obtain the target category corresponding to the image to be identified.

2. The method according to claim 1, characterized in that The step of selecting target image features required in the target task scenario from the image features includes: Acquire a feature application subset associated with the target task scenario; wherein the feature application subset includes a feature application score corresponding to each of the image features, and the feature application score is used to measure the importance of the image feature in the target task scenario; A target image feature is selected from the image features according to the feature application score corresponding to each of the image features.

3. The method according to claim 1, characterized in that The method further comprises: In the process of training the image classification model under the target task scenario, the average distance between two training samples of different categories under the target feature group, the average distance between two training samples of the same category under the target feature group, and the mean of the classification confidence of each training sample output by the linear classifier are determined; wherein the target feature group refers to any one of the multiple groups of image features extracted from the training samples by the multiple pre-trained deep learning networks; The feature application score corresponding to the target feature group is determined according to the average distance between the pairwise training samples of different categories under the target feature group, the average distance between the pairwise training samples of the same category under the target feature group, and the mean of the classification confidence of each training sample output by the linear classifier.

4. The method according to claim 3, characterized in that The step of determining the average distance between two training samples of different categories under the target feature group includes: According to the category of each of the training samples, each pair of training samples of different categories is formed into sample pairs, and the number of the sample pairs is determined; If it is determined that the target feature group is the jth group of image features extracted from the training samples by the multiple pre-trained deep learning networks, then for each of the sample pairs, the following operations are performed: extracting the jth group of image features from the two training samples included in the sample pair respectively; determining the distance between the jth group of image features of the two training samples; According to the number of sample pairs and the distance between the jth group of image features of two training samples in each sample pair, the average distance between two training samples of different categories when the target feature group is the jth group of image features is determined.

5. The method according to claim 4, characterized in that Determining the average distance between two training samples of the same category under the target feature group includes: According to the category of each of the training samples, each pair of training samples of the same category is grouped into sample pairs, and the number of the sample pairs is determined; If it is determined that the target feature group is the jth group of image features extracted from the training samples by the multiple pre-trained deep learning networks, then for each of the sample pairs, the following operations are performed: extracting the jth group of image features from the two training samples included in the sample pair respectively; determining the distance between the jth group of image features of the two training samples; According to the number of sample pairs and the distance between the jth group of image features of two training samples in each sample pair, the average distance between two training samples of the same category when the target feature group is the jth group of image features is determined.

6. The method according to claim 5, characterized in that Determining the feature application score corresponding to the target feature group according to the average distance between the two training samples of different categories under the target feature group, the average distance between the two training samples of the same category under the target feature group, and the mean of the classification confidence of each training sample output by the linear classifier includes: Obtaining the weight coefficients corresponding to the average distances of the pairwise training samples of different categories under the target feature group, the average distances of the pairwise training samples of the same category under the target feature group, and the mean values ​​of the classification confidences of the training samples output by the linear classifier; According to the weight coefficient, the average distance between the pairwise training samples of different categories under the target feature group, the average distance between the pairwise training samples of the same category under the target feature group, and the classification confidence of each training sample output by the linear classifier are weighted and summed, and the weighted sum result is used as the feature application score corresponding to the target feature group.

7. The method according to claim 1, characterized in that The multiple pre-trained deep learning networks include an ImageNet classification network, an image edge extraction network, and an image texture classification network; the linear classifier is a logistic regression classifier.

8. An image classification device, characterized in that: The apparatus is configured in an embedded device, wherein an image classification model composed of a plurality of pre-trained deep learning networks and linear classifiers is deployed in the embedded device, and the training process of the image classification model in a target task scenario is completed in the embedded device, and the apparatus comprises: An extraction module, used to obtain an image to be identified in a target task scenario, and extract image features from the image to be identified through the multiple pre-trained deep learning networks; An adaptive module, used to select target image features required in the target task scenario from the image features; The classification module is used to classify and identify the target image features through the linear classifier to obtain the target category corresponding to the image to be identified.

9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images