Image classification method, image classification apparatus, and electronic device

Abstract

The application discloses an image classification method, an image classification device and electronic equipment, and belongs to the field of image classification. The image classification method provided by the application comprises the following steps: acquiring a target image, wherein the target image is a to-be-classified image belonging to a target domain; inputting the target image into a second image classification model; wherein the second image classification model is an image classification model obtained by adjusting a first image classification model based on target domain feature information corresponding to the target domain; the first image classification model is a trained image classification model and is used for classifying images belonging to a source domain; and outputting a classification result of the target image.

Description

Technical Field

[0001] This application belongs to the field of image classification, specifically relating to an image classification method, an image classification device, and an electronic device. Background Technology

[0002] In the field of image classification, images from the source domain are generally used as training samples to train image classification models. However, the limited number of training samples makes it difficult to cover images of all styles. In the application of image classification models, if a well-trained model is used to identify images in the target domain that differ significantly from the source domain, the recognition accuracy will be low.

[0003] To address the aforementioned issues, related technologies typically involve first inputting the image to be recognized, belonging to the target domain, into a pre-trained domain transfer model to transform it into an image to be recognized in the source domain; then, the image to be recognized in the source domain is input into a pre-trained image classification model to achieve image classification.

[0004] However, in related technologies, the classification speed is relatively slow because the image to be identified needs to be preprocessed by a domain transfer model and then classified by an image classification model. Summary of the Invention

[0005] This application provides an image classification method, an image classification device, and an electronic device, which can solve the problem of slow image classification speed in related technologies.

[0006] In a first aspect, embodiments of this application provide an image classification method, the method comprising:

[0007] Acquire a target image, wherein the target image is an image to be classified that belongs to the target domain;

[0008] The target image is input into the second image classification model; wherein, the second image classification model is an image classification model obtained by adjusting the first image classification model based on the target domain feature information corresponding to the target domain; the first image classification model is a pre-trained image classification model, and is used to classify images belonging to the source domain;

[0009] Output the classification result of the target image.

[0010] Secondly, embodiments of this application provide an image classification device, including: an acquisition module, an input module, and an output module;

[0011] The acquisition module is used to acquire a target image, wherein the target image is an image to be classified that belongs to the target domain;

[0012] The input module is used to input the target image into the second image classification model; wherein, the second image classification model is an image classification model obtained by adjusting the first image classification model based on the target domain feature information corresponding to the target domain; the first image classification model is a pre-trained image classification model, and is used to classify images belonging to the source domain;

[0013] The output module is used to output the classification result of the target image.

[0014] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0015] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0016] In this embodiment, a target image is acquired, which is an image to be classified belonging to a target domain; the target image is input into a second image classification model; wherein, the second image classification model is an image classification model obtained by adjusting a first image classification model based on target domain feature information corresponding to the target domain; the first image classification model is a pre-trained image classification model used to classify images belonging to the source domain; and the classification result of the target image is output. Thus, since the second image classification model is an image classification model obtained by adjusting the first image classification model based on target domain feature information corresponding to the target domain, the second image classification model can be applied to the classification of images belonging to the target domain. Compared with related technologies, there is no need to input the image to be classified in the target domain into a domain transfer model for preprocessing; the image to be classified in the target domain is directly classified by the second image classification model, resulting in faster image classification speed and solving the problem of slow image classification speed in related technologies. Attached Figure Description

[0017] Figure 1 This is a schematic flowchart of an image classification method in related technologies;

[0018] Figure 2 This is a schematic flowchart of an image classification method provided in an embodiment of this application;

[0019] Figure 3 This is a schematic flowchart of another image classification method provided in the embodiments of this application;

[0020] Figure 4This is a schematic flowchart of another image classification method provided in the embodiments of this application;

[0021] Figure 5 This is a schematic flowchart of another image classification method provided in the embodiments of this application;

[0022] Figure 6 This is a schematic flowchart of another image classification method provided in the embodiments of this application;

[0023] Figure 7 This is a schematic flowchart of another image classification method provided in the embodiments of this application;

[0024] Figure 8 This is a schematic flowchart illustrating a training method for a target classification model provided in an embodiment of this application;

[0025] Figure 9 This is a schematic flowchart illustrating another training method for a target classification model provided in an embodiment of this application;

[0026] Figure 10 This is a schematic structural diagram of an image classification device provided in an embodiment of this application;

[0027] Figure 11 This is a schematic structural diagram of an electronic device provided in an embodiment of this application;

[0028] Figure 12 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0030] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0031] The applicant noted that in the relevant technology, such as Figure 1 As shown, to address the significant discrepancy between the image scenes used in training and applying image classification models, a common approach is to first input the target image (the image to be identified in the target domain) into a pre-trained domain transfer model to transform it into an image in the source domain. Then, the image in the source domain is input into a pre-trained first image classification model to predict and classify the image. This first image classification model can be trained using image samples from the source domain, making it suitable for classifying images belonging to the source domain. However, this process, where the target image is sequentially classified through the domain transfer model and the first image classification model, results in a relatively slow image classification speed.

[0032] Based on this, such as Figure 2 As shown, the image classification method provided in this application can adjust the first image classification model using target domain feature information corresponding to the target domain to obtain a second image classification model suitable for classifying images belonging to the target domain. In this way, the target image can be directly classified using the second image classification model. Compared with related technologies, the target image does not need to be preprocessed using a domain transfer model, resulting in faster image classification speed.

[0033] The image classification method, image classification device, and electronic device provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0034] Figure 3 This is a schematic flowchart of an image classification method provided in an embodiment of this application.

[0035] like Figure 3 As shown, the image classification method provided in this application embodiment may include:

[0036] Step 310: Obtain the target image, which is the image to be classified belonging to the target domain;

[0037] Step 320: Input the target image into the second image classification model; wherein, the second image classification model is an image classification model obtained by adjusting the first image classification model based on the target domain feature information corresponding to the target domain; the first image classification model is a pre-trained image classification model, and is used to classify images belonging to the source domain;

[0038] Step 330: Output the classification result of the target image.

[0039] In step 310, the target image is the image to be classified that belongs to the target domain, and there is a style difference between the image belonging to the target domain and the image belonging to the source domain.

[0040] In step 320, the first image classification model can be an image classification model trained using image samples from the source domain as training samples. Therefore, the first image classification model is suitable for classifying images belonging to the source domain, but less suitable for classifying images belonging to the target domain. Based on this, the second image classification model is obtained by adjusting the first image classification model based on target domain feature information corresponding to the target domain. Target domain feature information can characterize features unique to the target domain and is used to distinguish style differences between images belonging to the target domain and images belonging to the source domain. Therefore, the image classification model obtained by adjusting the first image classification model based on target domain feature information corresponding to the target domain is suitable for classifying images belonging to the target domain.

[0041] In step 330, this embodiment of the application can employ a second image classification model to directly classify the target image. Furthermore, since the second image classification model is obtained by adjusting the first image classification model based on target domain feature information corresponding to the target domain, the second image classification model is applicable to the classification of images belonging to the target domain. Images to be classified belonging to the target domain can be directly input into the second image classification model, and the classification result of the images to be classified is output. Compared with related technologies, there is no need to preprocess the images to be classified in the target domain using a domain transfer model, which facilitates the direct classification of images in the target domain through the second image classification model, resulting in faster image classification speed and solving the problem of slow image classification speed in related technologies.

[0042] According to the image classification method provided in this application, a target image is acquired, which is an image to be classified belonging to a target domain; the target image is input into a second image classification model; wherein, the second image classification model is an image classification model obtained by adjusting a first image classification model based on target domain feature information corresponding to the target domain; the first image classification model is a pre-trained image classification model used to classify images belonging to the source domain; and the classification result of the target image is output. Thus, since the second image classification model is an image classification model obtained by adjusting the first image classification model based on target domain feature information corresponding to the target domain, the second image classification model can be applied to the classification of images belonging to the target domain. Compared with related technologies, there is no need to input the image to be classified in the target domain into a domain transfer model for preprocessing; the image to be classified in the target domain is directly classified by the second image classification model, resulting in faster image classification speed and solving the problem of slow image classification speed in related technologies.

[0043] In one specific embodiment, to improve the classification speed of the target image, a second image classification model can be pre-trained before acquiring and classifying the target image, so that the second image classification model can be directly used to classify the target image subsequently. An example is given below.

[0044] Figure 4 This is a schematic flowchart of another image classification method provided in the embodiments of this application.

[0045] like Figure 4 As shown, the image classification method provided in this application embodiment may include:

[0046] Step 410: Obtain the first image classification model;

[0047] The first image classification model is a pre-trained image classification model used to classify images belonging to the source domain.

[0048] Step 420: Obtain target domain feature information corresponding to the target domain, wherein the target domain feature information is used to characterize the features unique to the target domain;

[0049] Step 430: Adjust the first image classification model based on the target domain feature information to obtain a second image classification model, wherein the second image classification model is suitable for classifying images belonging to the target domain;

[0050] Step 440: Obtain the target image, which is the image to be classified belonging to the target domain;

[0051] Step 450: Input the target image into the second image classification model;

[0052] Step 460: Output the classification result of the target image.

[0053] Steps 410 to 430 can be performed before step 440.

[0054] In particular, step 440 can refer to the specific content of step 310, step 450 can refer to the specific content of step 320, and step 460 can refer to the specific content of step 330.

[0055] In step 410, the first image classification model can be an image classification model trained using image samples from the source domain as training samples. Therefore, the first image classification model can be applied to the classification of images belonging to the source domain. The first image classification model can be a neural network learning model.

[0056] The structure of the first image classification model can be as follows: Figure 7The five convolutional layers shown and the fully connected layers for classification prediction can be composed of a VGGNet (Visual Geometry Group Network) structure, or a ResNet (Residual Network) structure, etc. This application does not impose any specific limitations.

[0057] Understandably, in practical applications, to alleviate the generalization problem of image classification models, when collecting training samples for the first image classification model, it's common practice to collect as many and as diverse a range of training samples as possible, ensuring the training samples closely match the distribution used in real-world applications. However, images are incredibly diverse, and training samples rarely cover all image distributions. Consequently, there's a discrepancy between the data distribution of the training images (corresponding to the source domain) and the data distribution of the images to be identified (corresponding to the target domain), leading to lower accuracy for the first image classification model in recognizing images belonging to the target domain. In other words, the first image classification model is not well-suited for classifying images belonging to the target domain.

[0058] In step 420, the target domain feature information corresponding to the target domain is used to distinguish the style differences between images belonging to the target domain and images belonging to the source domain. The target domain feature information can be used to characterize features unique to the target domain. These domain-specific features can be predefined features based on actual usage requirements.

[0059] For example, domain-specific features may include at least one of the following: illumination, the proportion of an object in the image, color gradation, etc. Target domain feature information can be used to characterize the domain-specific features of the target domain. Target domain feature information may be high exposure, an object occupying less than 50% of the image, and a black and white tone; source domain feature information may be low exposure, an object occupying more than 70% of the image, and a color tone. Target domain feature information can distinguish the stylistic differences between images belonging to the target domain and images belonging to the source domain.

[0060] In step 420, the method of obtaining the target domain feature information is not limited. For example, the target domain feature information may be target domain feature information extracted in real time from an image belonging to the target domain, or it may be pre-prepared target domain feature information received from other terminals. This application does not impose any specific limitations.

[0061] In step 430, the first image classification model is adjusted based on the target domain feature information to obtain a second image classification model, making the second image classification model suitable for classifying images belonging to the target domain. The second image classification model can be a neural network learning model.

[0062] In steps 440 to 460, after obtaining the second image classification model, a target image (the image to be classified belonging to the target domain) can be input into the second image classification model, and the classification result of the target image is output. It is understood that this application can directly input the image to be classified belonging to the target domain into the second image classification model and output the classification result of the image to be classified. Compared with related technologies, there is no need to preprocess the image to be classified in the target domain by inputting it into a domain transfer model, which is beneficial for the image to be classified in the target domain to be directly classified through the second image classification model, resulting in faster image classification speed and solving the problem of slow image classification speed in related technologies.

[0063] According to the image classification method provided in this application embodiment, a first image classification model is obtained, wherein the first image classification model is a pre-trained image classification model used to classify images belonging to the source domain; target domain feature information corresponding to the target domain is obtained, wherein the target domain feature information is used to characterize features unique to the target domain; the first image classification model is adjusted based on the target domain feature information to obtain a second image classification model, wherein the second image classification model is suitable for classifying images belonging to the target domain; a target image is obtained, wherein the target image is an image to be classified belonging to the target domain; the target image is input into the second image classification model; and the classification result of the target image is output. Thus, by adjusting the first image classification model based on the target domain feature information corresponding to the target domain, the obtained second image classification model can be applied to the classification of images belonging to the target domain. Compared with related technologies, there is no need to input the image to be classified in the target domain into a domain transfer model for preprocessing; the image to be classified in the target domain is directly classified through the second image classification model, resulting in faster image classification speed and solving the problem of slow image classification speed in related technologies.

[0064] Figure 5 This could be a schematic flowchart of an image classification method provided in another embodiment of this application. Figure 5 It is possible Figure 4 Based on the illustrated embodiment, the process of adjusting the first image classification model based on target domain feature information is further defined.

[0065] In one specific embodiment, to make the second image classification model more suitable for classifying images belonging to the target domain, such as Figure 5 As shown, in step 430 above, adjusting the first image classification model based on the target domain feature information to obtain the second image classification model includes:

[0066] Step 4301: Obtain source domain feature information corresponding to the source domain in the first image classification model, wherein the source domain feature information is used to characterize features unique to the source domain;

[0067] Step 4302: Replace the source domain feature information in the first image classification model with the target domain feature information to obtain the adjusted image classification model;

[0068] Step 4303: Based on the adjusted image classification model, obtain the second image classification model.

[0069] In step 4301, since source domain feature information can be used to characterize features unique to the source domain, style features unique to the source domain image can be derived based on the source domain feature information. For images belonging to both the source and target domains, the source domain feature information of an image belonging to the source domain can differ from the target domain feature information of an image belonging to the target domain. Therefore, there can often be significant style differences between images belonging to the source domain and images belonging to the target domain.

[0070] For example, domain-specific features can include information about image dimensions. Source domain features can be features of a three-dimensional image. Target domain features can be features of a two-dimensional image. Therefore, there can be significant stylistic differences between images in the target domain and those in the source domain.

[0071] In step 4301, there are no restrictions on the method of obtaining source domain feature information. For example, the source domain feature information may be source domain feature information extracted from the first image classification model in real time, or source domain feature information extracted from the first image classification model in advance, or source domain feature information extracted after processing the training image samples of the first image classification model using a trained domain classification model, etc. This application does not impose any specific restrictions.

[0072] In step 4302, after replacing the source domain feature information in the first image classification model with the target domain feature information, the resulting adjusted image classification model can be applied to the classification of images belonging to the target domain. It is understood that, because the source domain feature information in the first image classification model is replaced, the adjusted image classification model (or the second image classification model) can change from being applicable to the classification of images belonging to the source domain to being applicable to the classification of images belonging to the target domain.

[0073] In step 4303, obtaining a second image classification model based on the adjusted image classification model may include: using the adjusted image classification model as the second image classification model. This reduces the difficulty of obtaining the second image classification model while ensuring its applicability to the classification of images belonging to the target domain.

[0074] Alternatively, in step 4303, obtaining a second image classification model based on the adjusted image classification model may include: training the adjusted image classification model using image samples belonging to the target domain to obtain the second image classification model. It is understood that, to avoid the target domain feature information obtained in practical applications failing to adequately represent the unique information of the target domain, after obtaining the adjusted image classification model, this application may also use a small number of image samples belonging to the target domain as training samples to train the adjusted image classification model to obtain the second image classification model, thereby further improving the accuracy of the second image classification model in recognizing target images.

[0075] It should be understood that the process of step 430 is not limited to the processes listed above. In fact, in the embodiments of this application, any process that adjusts the first image classification model based on the target domain feature information to obtain a second image classification model suitable for classifying images in the target domain is acceptable. The replacement operation described above is only an example. For example, in some cases, the source domain feature information may not be replaced, but rather it may be weakened, for example, by reducing the weight or importance of the source domain feature information to a lower level, while the target domain feature information may be strengthened, for example, by increasing the weight or importance of the target domain feature information to a higher level. In the embodiments of this application, "higher" and "lower" are only relative concepts. "Higher" may be, for example, greater than 50% to less than 99%, and "lower" may be greater than 1% to less than 50%, etc.

[0076] In the image classification method provided in this application embodiment, source domain feature information corresponding to the source domain in the first image classification model is obtained. This source domain feature information is used to characterize features unique to the source domain. The source domain feature information in the first image classification model is replaced with the target domain feature information to obtain an adjusted image classification model. Based on the adjusted image classification model, a second image classification model is obtained. Thus, by replacing the source domain feature information in the first image classification model with the target domain feature information, the second image classification model can be better applied to the classification of images belonging to the target domain.

[0077] In a specific embodiment, the source domain feature information in the first image classification model can be in the form of feature vectors. That is, in step 4301, the source domain feature information may include N specified feature vectors, and the target domain feature information may include N target feature vectors, where N is an integer and N≥1. For example, N can be 1, 2, or 3, etc., and this application does not impose specific limitations. It should be understood that in this embodiment, the source domain feature information being in the form of feature vectors is merely an example. In some cases, the source domain feature information can also be in other forms. For example, in some cases, the source domain feature information can be in forms other than feature vectors, such as custom strings or other forms. In this embodiment, the forms of source domain feature information and target domain feature information are not specifically limited; the emphasis is on the fact that the source domain feature information can represent features unique to the source domain, and the target domain feature information can represent features unique to the target domain.

[0078] Accordingly, in step 4302 above, replacing the source domain feature information in the first image classification model with the target domain feature information to obtain the adjusted image classification model includes:

[0079] The N specified feature vectors in the first image classification model are replaced with the N target feature vectors to obtain the adjusted image classification model.

[0080] In this embodiment, the number of target feature vectors can be determined based on the number of specified feature vectors included in the source domain feature information. Specifically, the number of specified feature vectors included in the source domain feature information can be determined to be N. Based on this, N target feature vectors corresponding to the N specified feature vectors are obtained as target domain feature information. This allows all specified feature vectors included in the source domain feature information to be replaced one by one with target feature vectors, thus avoiding the presence of unreplaced source domain feature information in the adjusted image classification model, which would affect the accuracy of the second image classification model.

[0081] In practical applications, embodiments of this application can replace the N specified feature vectors in the first image classification model according to the specific structure of the first image classification model. For example, the first image classification model may include: N specified convolutional layers and a first prediction layer for classification prediction, wherein the Nth specified convolutional layer is connected to the first prediction layer; the i-th specified feature vector in the N specified feature vectors corresponds to the i-th specified convolutional layer in the N specified convolutional layers, where i is an integer and N≥i≥1.

[0082] It is understood that the first image classification model may include L (L > N) convolutional layers. The first LN unspecified convolutional layers near the input are generally used to extract local feature information of the input image, while the last N specified convolutional layers near the first prediction layer are generally used to extract overall feature information of the input image. Domain feature information is generally associated with the overall feature information of the image. The first prediction layer may be a fully connected layer for classification prediction or other output layers; this application does not impose specific limitations.

[0083] In step 420 above, obtaining the target domain feature information corresponding to the target domain includes: using a target classification model to obtain the target domain feature information corresponding to the target domain, wherein the target classification model is a pre-trained domain classification model;

[0084] The target classification model includes: N target convolutional layers and a second prediction layer for classification prediction. The Nth target convolutional layer is connected to the second prediction layer. The i-th target convolutional layer corresponds to the i-th specified convolutional layer among the N specified convolutional layers. The i-th target feature vector among the N target feature vectors is output by the i-th target convolutional layer.

[0085] It is understood that the target classification model may include J (J > N) convolutional layers and a second prediction layer. The first JN non-target convolutional layers near the input are generally used to extract local feature information of the input image, while the last N target convolutional layers near the second prediction layer are generally used to extract overall feature information of the input image. Domain feature information is generally related to the overall features of the image. When the target classification model is a pre-trained domain classification model, the N target feature vectors output by the N target convolutional layers can be used as the target domain feature information corresponding to the target domain. The second prediction layer can be a fully connected layer used for classification prediction or other output layers; this application does not impose specific limitations.

[0086] Accordingly, replacing the N specified feature vectors in the first image classification model with the N target feature vectors includes:

[0087] For each of the N specified feature vectors, perform the following replacement operation: replace the i-th specified feature vector among the N specified feature vectors with the i-th target feature vector among the N target feature vectors; where 1≤i≤N;

[0088] In the case of N=1, the sum of the output of the first specified convolutional layer and the first target feature vector is used as the input of the first prediction layer;

[0089] When N≥2 and N-1≥q≥1, the output of the q-th specified convolutional layer and the sum of the q-th target feature vector are used as the input of the (q+1)-th specified convolutional layer, where q is an integer;

[0090] When N≥2 and q=N, the output of the q-th specified convolutional layer and the sum of the q-th target feature vector are used as the input of the first prediction layer.

[0091] In this way, based on the correspondence between the target convolutional layer in the target classification model and the specified convolutional layer in the first image classification model, the i-th specified feature vector among the N specified feature vectors can be replaced in an orderly manner with the i-th target feature vector among the N target feature vectors.

[0092] For example, such as Figure 7 As shown, L is 5, J is 5, and N is 3. The N specified convolutional layers can be the three specified convolutional layers closest to the first fully connected layer in an image classification model. The N target convolutional layers can be the three target convolutional layers closest to the second fully connected layer in a domain classification model. It should be understood that the values ​​of L and N are merely examples. In this embodiment, L can also take values ​​other than 5, such as 4, 3, or 6. N can also take values ​​other than 3, such as 2, 1, or 4.

[0093] like Figure 7 As shown, i can be 1, 2, or 3. The adjustment process of the first image classification model includes: replacing the first specified feature vector among the three specified feature vectors with the first target feature vector among the three target feature vectors; replacing the second specified feature vector among the three specified feature vectors with the second target feature vector among the three target feature vectors; replacing the third specified feature vector among the three specified feature vectors with the third target feature vector among the three target feature vectors, to obtain the adjusted image classification model, and then obtaining the second image classification model based on the adjusted image classification model.

[0094] Furthermore, such as Figure 7As shown, when the second image classification model is used to classify the target image, the target image is input into the second image classification model. The target image is the image to be identified that belongs to the target domain. Taking the target image as a data representation with a length * height * number of channels of 256 * 256 * 3 as an example, after the target image is processed by the first convolutional layer of the second image classification model for dimensionality reduction, it can output a feature vector for representing a 128 * 128 * 64 image. After the second convolutional layer of the second image classification model for dimensionality reduction, it can output a feature vector for representing a 64 * 64 * 128 image. After the first designated convolutional layer (i.e., the third convolutional layer) of the second image classification model for dimensionality reduction, it can output a feature vector for a 32 * 32 * 256 image. The feature vectors are processed as follows: the sum of the feature vectors output from the first specified convolutional layer and the first target feature vector is input into the second specified convolutional layer (i.e., the fourth convolutional layer) of the second image classification model for dimensionality reduction, and the output is a feature vector representing a 16*16*512 image; the sum of the feature vectors output from the second specified convolutional layer and the second target feature vector is input into the third specified convolutional layer (i.e., the fifth convolutional layer) of the second image classification model for dimensionality reduction, and the output is a feature vector representing an 8*8*1024 image; the sum of the feature vectors output from the third specified convolutional layer and the third target feature vector is input into the first fully connected layer of the second image classification model for classification, and the classification result of the target image is output.

[0095] Figure 6 This is a schematic flowchart of an image classification method provided in another embodiment of this application. Figure 6 It is possible Figure 4 Based on the illustrated embodiment, the process of acquiring target domain feature information is further defined.

[0096] The following describes in detail the process of obtaining target domain feature information using a target classification model. For example... Figure 6 As shown, in step 420 above, obtaining the target domain feature information corresponding to the target domain may include:

[0097] Step 4201: Obtain the target classification model, which is a pre-trained domain classification model;

[0098] Step 4202: Obtain M images belonging to the target domain;

[0099] Step 4203: Input the M images into the target classification model;

[0100] Step 4204: Process the M images using the target classification model to obtain target domain feature information corresponding to the target domain; where M is a positive integer.

[0101] The target classification model is a pre-trained domain classification model, which is suitable for classifying images belonging to various different domains. For example, the structure of a domain classification model can be as follows: Figure 7 The structure consists of five convolutional layers and a fully connected layer for classification prediction, as shown. Alternatively, it can be composed of VGGNet, ResNet, etc. This application does not impose any specific limitations.

[0102] Thus, embodiments of this application can use a target classification model to extract target domain feature information corresponding to the target domain from images belonging to the target domain, so as to accurately represent the information unique to the target domain.

[0103] In this embodiment of the application, in order to obtain more comprehensive target domain feature information, the target domain feature information may include one or more target feature vectors, depending on the total number of convolutional layers in the target classification model. In step 4204 above, processing the M images using the target classification model to obtain target domain feature information corresponding to the target domain includes:

[0104] The M images are processed by the target classification model to obtain N target feature vectors, where N is an integer and N≥1;

[0105] The N target feature vectors are used as target domain feature information corresponding to the target domain.

[0106] For example, if the target classification model includes 5 convolutional layers and a prediction layer for classification prediction, the feature vectors output by 1 to 3 convolutional layers closest to the prediction layer can be used as the target feature vectors. If the target classification model includes 500 convolutional layers and a prediction layer for classification prediction, the feature vectors output by 1 to 300 convolutional layers closest to the prediction layer can be used as the target feature vectors.

[0107] In this way, one or more target feature vectors can be flexibly obtained as target domain feature information, depending on the total number of convolutional layers in the target classification model.

[0108] For example, such as Figure 7 As shown, the feature vectors output by the three convolutional layers closest to the second prediction layer in the domain classification model can be used as target feature vectors.

[0109] In one specific embodiment, one of the N target feature vectors can correspond to the output of a target convolutional layer in the target classification model. Specifically, the target classification model may include: N target convolutional layers and a second prediction layer for classification prediction, wherein the Nth target convolutional layer is connected to the second prediction layer;

[0110] The process of processing the M images using the target classification model to obtain N target feature vectors includes:

[0111] The M target inputs corresponding to the M images are processed by the i-th target convolutional layer in the target classification model to obtain the i-th target feature vector among the N target feature vectors; where i is an integer and N≥i≥1.

[0112] like Figure 7 As shown, N is 3 for example, and the target classification model can include 3 target convolutional layers.

[0113] When i is 1, the M target inputs corresponding to the M images can be processed by the first target convolutional layer in the target classification model to obtain the first target feature vector among the three target feature vectors.

[0114] When i is 2, the M target inputs corresponding to the M images can be processed by the second target convolutional layer in the target classification model to obtain the second target feature vector among the three target feature vectors.

[0115] When i is 3, the M target inputs corresponding to the M images can be processed by the third target convolutional layer in the target classification model to obtain the third target feature vector among the three target feature vectors.

[0116] In this way, one target convolutional layer in the target classification model can output one target feature vector, and thus N target convolutional layers in the target classification model can output N target feature vectors.

[0117] In one specific embodiment, the number M of images input to the target classification model is an integer greater than or equal to 1. The larger M is, the more stable the obtained target feature vector. An example is given below.

[0118] When M equals 1, such as Figure 7As shown, an image belonging to the target domain is input into the domain classification model. The target domain image can be represented by data with length * height * number of channels of 256 * 256 * 3. After the target domain image is processed by the first convolutional layer for dimensionality reduction, the output is a feature vector representing a 128 * 128 * 64 image. After the second convolutional layer for dimensionality reduction, the output is a feature vector representing a 64 * 64 * 128 image. After the first target convolutional layer (i.e., the third convolutional layer) for dimensionality reduction, the output is a feature vector representing a 32 * 32 * 256 image. The feature vector output from the first target convolutional layer is input into the second target convolutional layer (i.e., the fourth convolutional layer) for dimensionality reduction, and the output is a feature vector representing a 16 * 16 * 512 image. The feature vector output from the second target convolutional layer is input into the third target convolutional layer (i.e., the fifth convolutional layer) for dimensionality reduction, and the output is a feature vector representing an 8 * 8 * 1024 image. The feature vector output from the third target convolutional layer is input into the second fully connected layer for classification processing, and the domain classification result is output.

[0119] In this embodiment, instead of using the domain classification result output by the domain classification model, the feature vectors output by the three convolutional layers closest to the second fully connected layer out of the five convolutional layers of the domain classification model are obtained. Furthermore, the above-mentioned processing of the M target inputs corresponding to the M images by the i-th target convolutional layer in the target classification model to obtain the i-th target feature vector among the N target feature vectors can include: using the feature vector output by the first target convolutional layer as the first target feature vector, the feature vector output by the second target convolutional layer as the second target feature vector, and the feature vector output by the third target convolutional layer as the third target feature vector.

[0120] When M is greater than 1, the step of processing the M target inputs corresponding to the M images through the i-th target convolutional layer in the target classification model to obtain the i-th target feature vector among the N target feature vectors includes:

[0121] The i-th target convolutional layer processes the M target inputs corresponding to the M images respectively to extract M domain feature vectors, wherein one of the M domain feature vectors corresponds to one of the M target inputs;

[0122] Calculate the average value of the M domain feature vectors, and use the average value of the M domain feature vectors as the i-th target feature vector among the N target feature vectors, where M≥2.

[0123] In this way, since each of the N target feature vectors is obtained by averaging the M domain feature vectors, the stability is good and it can accurately represent the information unique to the target domain.

[0124] like Figure 7 As shown, in cases where N equals 3 and M equals 10, the operation of inputting an image belonging to the target domain into the domain classification model 10 times can be performed. The average of the 10 domain feature vectors output by the first target convolutional layer is then used as the first target feature vector, the average of the 10 domain feature vectors output by the second target convolutional layer is used as the second target feature vector, and the average of the 10 domain feature vectors output by the third target convolutional layer is used as the third target feature vector. Since each of the 10 target feature vectors is calculated by averaging, the stability is good, and it can accurately represent the information unique to the target domain. It should be understood that the embodiments of this application may also not use the method of directly taking the average of multiple domain feature vectors, but instead select a specific domain feature vector that meets the set conditions. Depending on the set conditions, there can be one or more specific domain feature vectors. When there are multiple specific domain feature vectors, the average of these multiple specific domain feature vectors can be taken. This ensures that the domain feature vector is selected according to the requirements, thereby better meeting the continuously changing needs.

[0125] In another specific embodiment, to ensure that the target classification model has good domain classification ability and can distinguish domain features of different domains while keeping the image category unchanged, such as... Figure 8 As shown, the training process of the target classification model may include:

[0126] Step 810: Obtain K training datasets, where each of the K training datasets corresponds to an image category, and the image categories corresponding to the K training datasets are different; each training dataset includes multiple images belonging to different domains, where K is an integer and K≥2;

[0127] Step 820: Obtain the initial domain classification model;

[0128] Step 830: Train the initial domain classification model using the K training datasets to obtain the target classification model.

[0129] In this way, since multiple images in a training dataset belong to different domains but have the same image category, using this kind of training dataset to train the initial domain classification model can treat the same domain as a category for domain classification training. Furthermore, since the images in a training dataset have the same image category, the training can be free from interference from image category and can learn the differences between domains, so that the target classification model can be used to extract the differences between domains (i.e., domain feature information) in the future.

[0130] For example, in the embodiments of this application, the initial domain classification model can be trained successively using K training datasets. Figure 9 As shown, in step 830 above, training the initial domain classification model using the K training datasets to obtain the target classification model includes:

[0131] Step 8301: Train the initial domain classification model using the first training dataset from the K training datasets to obtain the first domain classification model;

[0132] Step 8302: Train the (s-1)th domain classification model using the s-th training dataset from the K training datasets to obtain the s-th domain classification model, where s is a positive integer and K≥s≥2;

[0133] Step 8303: When s = K, obtain the Kth domain classification model and use the Kth domain classification model as the target classification model.

[0134] For example, when K equals 3, the initial domain classification model is trained using the first training dataset out of the three training datasets to obtain the first domain classification model; the first domain classification model is trained using the second training dataset out of the three training datasets to obtain the second domain classification model; the second domain classification model is trained using the third training dataset out of the three training datasets to obtain the third domain classification model; and the third domain classification model is used as the target classification model.

[0135] In this way, the initial domain classification model is trained by using K training datasets in succession. The more training datasets there are, the more accurately the trained target classification model can learn the differences between domains.

[0136] Specifically, when training the target classification model using a single training dataset, in step 3302, the s-th training dataset includes P images belonging to different domains, where P is a positive integer and P≥2;

[0137] In step 8302 above, training the classification model for the (s-1)th domain using the s-th training dataset from the K training datasets to obtain the classification model for the s-th domain may include:

[0138] The first image among the P images is input into the (s-1)th domain classification model to obtain a first predicted value corresponding to the first image; based on the first predicted value and the first ground truth value, a first contrastive loss value is obtained; wherein, the first ground truth value is obtained based on the first image;

[0139] The j-th image from the P images is input into the (s-1)-th domain classification model to obtain the j-th predicted value corresponding to the j-th image; based on the j-th predicted value and the j-th ground truth value, the j-th contrastive loss value is obtained; wherein the j-th ground truth value is obtained based on the j-th image; wherein j is a positive integer, and P≥j≥2;

[0140] Based on the first contrastive loss value, ... the j-th contrastive loss value, the parameters of the (s-1)-th domain classification model are adjusted to obtain the s-th domain classification model.

[0141] In this way, the classification model for the (s-1)th domain is trained using P images from the s-th training dataset. The more images in the training dataset, the more accurately the trained target classification model can learn the differences between domains.

[0142] Furthermore, when training the target classification model using the training dataset, the center loss function can be used to minimize the intra-class (domain) distance while ensuring that domain-specific features are separable, thereby increasing the distance between classes (domains) and accurately extracting inter-domain differences (i.e., domain-specific information).

[0143] The image classification method provided in this application embodiment is described below with reference to practical application scenarios.

[0144] like Figure 7 As shown, in a practical application scenario, taking a training dataset of three or more, with each training dataset containing 10 images as an example, the process of the image classification method provided in this application embodiment may include:

[0145] Obtain three or more training datasets, where one of the training datasets corresponds to an image category, and the image categories corresponding to the three or more training datasets are different; each training dataset includes multiple images belonging to different domains;

[0146] Obtain the initial domain classification model;

[0147] The initial domain classification model is trained using the three or more training datasets to obtain a trained domain classification model.

[0148] Ten images belonging to the target domain are sequentially input into a trained domain classification model. The domain feature vectors output by the 3rd, 4th, and 5th convolutional layers of the domain classification model are extracted. The average of the ten domain feature vectors output by the 3rd convolutional layer of the domain classification model is used as the first target feature vector, the average of the ten domain feature vectors output by the 4th convolutional layer of the domain classification model is used as the second target feature vector, and the average of the ten domain feature vectors output by the 5th convolutional layer of the domain classification model is used as the third target feature vector.

[0149] The first specified feature vector in the first image classification model is replaced with the first target feature vector, the second specified feature vector in the first image classification model is replaced with the second target feature vector, and the third specified feature vector in the first image classification model is replaced with the third target feature vector, resulting in the adjusted image classification model.

[0150] The adjusted image classification model is trained using a small number of image samples belonging to the target domain to obtain the second image classification model;

[0151] Input the target image into the second image classification model. The target image is the image to be identified that belongs to the target domain. The target image can be represented by data with a length * height * number of channels of 256 * 256 * 3.

[0152] The target image is processed by the first convolutional layer of the second image classification model, which outputs a feature vector representing a 128*128*64 image. It is then processed by the second convolutional layer of the second image classification model, which outputs a feature vector representing a 64*64*128 image. Finally, it is processed by the first designated convolutional layer (i.e., the third convolutional layer) of the second image classification model, which outputs a feature vector representing a 32*32*256 image. The sum of the feature vector output from the first designated convolutional layer and the first target feature vector is input into the second image classification model. After dimensionality reduction by two designated convolutional layers (i.e., the fourth convolutional layer), the output is a feature vector representing a 16*16*512 image. The sum of the feature vector output by the second designated convolutional layer and the second target feature vector is input into the third designated convolutional layer (i.e., the fifth convolutional layer) of the second image classification model for dimensionality reduction, and the output is a feature vector representing an 8*8*1024 image. The sum of the feature vector output by the third designated convolutional layer and the third target feature vector is input into the first fully connected layer of the second image classification model for classification, and the classification result of the target image is output.

[0153] In this way, by adjusting the first image classification model based on the three target feature vectors corresponding to the target domain, the resulting second image classification model can be applied to the classification of images belonging to the target domain. Compared with related technologies, there is no need to input the image to be classified in the target domain into the domain transfer model for preprocessing. The image to be classified in the target domain is directly classified by the second image classification model, resulting in faster image classification speed.

[0154] The image classification method provided in this application can be executed by an image classification device. This application uses an image classification device executing the image classification method as an example to illustrate the image classification device provided in this application.

[0155] Figure 10 This is a schematic structural diagram of an image classification device provided in an embodiment of this application.

[0156] like Figure 10 As shown, the image classification device 1000 provided in this application embodiment may include:

[0157] The module consists of an acquisition module 1001, an input module 1002, and an output module 1003.

[0158] The acquisition module 1001 is used to acquire a target image, wherein the target image is a to-be-classified image belonging to the target domain;

[0159] The input module 1002 is used to input the target image into the second image classification model; wherein, the second image classification model is an image classification model obtained by adjusting the first image classification model based on the target domain feature information corresponding to the target domain; the first image classification model is a pre-trained image classification model and is used to classify images belonging to the source domain;

[0160] The output module 1003 is used to output the classification result of the target image.

[0161] The image classification apparatus provided in this application includes an acquisition module, an input module, and an output module. The acquisition module is used to acquire a target image, which is an image to be classified belonging to a target domain. The input module is used to input the target image into a second image classification model. The second image classification model is an image classification model obtained by adjusting a first image classification model based on target domain feature information corresponding to the target domain. The first image classification model is a pre-trained image classification model used to classify images belonging to the source domain. The output module is used to output the classification result of the target image. Thus, since the second image classification model is obtained by adjusting the first image classification model based on target domain feature information corresponding to the target domain, the second image classification model is applicable to the classification of images belonging to the target domain. Compared with related technologies, there is no need to preprocess the image to be classified in the target domain by inputting it into a domain transfer model; the image to be classified in the target domain is directly classified by the second image classification model, resulting in faster image classification speed and solving the problem of slow image classification speed in related technologies.

[0162] Optionally, the image classification apparatus provided in this application embodiment further includes a processing module;

[0163] Before the acquisition module acquires the target image, the acquisition module is further configured to: acquire a first image classification model; acquire target domain feature information corresponding to the target domain, wherein the target domain feature information is used to characterize features unique to the target domain;

[0164] The processing module is used to adjust the first image classification model based on the target domain feature information to obtain a second image classification model, wherein the second image classification model is suitable for classifying images belonging to the target domain.

[0165] In this way, by adjusting the first image classification model based on the target domain feature information corresponding to the target domain, the resulting second image classification model can be applied to the classification of images belonging to the target domain. Furthermore, the second image classification model can be pre-trained before acquiring and classifying the target image, so that the second image classification model can be directly used to classify the target image in the subsequent process, thereby improving the classification speed of the target image.

[0166] Optionally, in the image classification apparatus provided in this application embodiment, during the process of adjusting the first image classification model based on the target domain feature information to obtain a second image classification model, the processing module is specifically used to: obtain source domain feature information corresponding to the source domain in the first image classification model, wherein the source domain feature information is used to characterize features unique to the source domain; replace the source domain feature information in the first image classification model with the target domain feature information to obtain an adjusted image classification model; and obtain a second image classification model based on the adjusted image classification model.

[0167] In this way, by replacing the source domain feature information in the first image classification model with the target domain feature information, the resulting second image classification model can be better applied to the classification of images belonging to the target domain.

[0168] Optionally, in the image classification apparatus provided in this application embodiment, the source domain feature information includes N specified feature vectors, and the target domain feature information includes N target feature vectors, where N is an integer and N≥1;

[0169] In the process of replacing the source domain feature information in the first image classification model with the target domain feature information to obtain the adjusted image classification model, the processing module 403 is specifically used to: replace N specified feature vectors in the first image classification model with the N target feature vectors to obtain the adjusted image classification model.

[0170] In this way, N target feature vectors corresponding to N specified feature vectors are obtained as target domain feature information, so that all specified feature vectors contained in the source domain feature information can be replaced one by one with target feature vectors. This avoids the existence of unreplaced source domain feature information in the adjusted image classification model, which would affect the accuracy of the second image classification model.

[0171] Optionally, in the image classification apparatus provided in this application embodiment, the first image classification model includes: N specified convolutional layers and a first prediction layer for classification prediction, wherein the Nth specified convolutional layer is connected to the first prediction layer; the i-th specified feature vector among the N specified feature vectors corresponds to the i-th specified convolutional layer among the N specified convolutional layers, where i is an integer and N≥i≥1;

[0172] In the process of acquiring target domain feature information corresponding to the target domain, the acquisition module is used to: acquire target domain feature information corresponding to the target domain using a target classification model, wherein the target classification model is a pre-trained domain classification model;

[0173] The target classification model includes: N target convolutional layers and a second prediction layer for classification prediction; the Nth target convolutional layer is connected to the second prediction layer; the i-th target convolutional layer corresponds to the i-th specified convolutional layer among the N specified convolutional layers; the i-th target feature vector among the N target feature vectors is output by the i-th target convolutional layer.

[0174] In the process of replacing the N specified feature vectors in the first image classification model with the N target feature vectors, the processing module is specifically used to: for each feature vector among the N specified feature vectors, perform the following replacement operation: replace the i-th specified feature vector among the N specified feature vectors with the i-th target feature vector among the N target feature vectors; where 1≤i≤N;

[0175] In the case of N=1, the sum of the output of the first specified convolutional layer and the first target feature vector is used as the input of the first prediction layer;

[0176] When N≥2 and N-1≥q≥1, the output of the q-th specified convolutional layer and the sum of the q-th target feature vector are used as the input of the (q+1)-th specified convolutional layer, where q is an integer;

[0177] When N≥2 and q=N, the output of the q-th specified convolutional layer and the sum of the q-th target feature vector are used as the input of the first prediction layer.

[0178] In this way, based on the correspondence between the target convolutional layer in the target classification model and the specified convolutional layer in the first image classification model, the i-th specified feature vector among the N specified feature vectors can be replaced in an orderly manner with the i-th target feature vector among the N target feature vectors.

[0179] Optionally, in the image classification apparatus provided in this application embodiment, during the process of acquiring target domain feature information corresponding to the target domain, the acquisition module is specifically used for:

[0180] Obtain the target classification model, which is a pre-trained domain classification model;

[0181] Obtain M images belonging to the target domain;

[0182] The M images are input into the target classification model;

[0183] The M images are processed by the target classification model to obtain target domain feature information corresponding to the target domain; where M is a positive integer.

[0184] Thus, embodiments of this application can use a target classification model to extract target domain feature information corresponding to the target domain from images belonging to the target domain, so as to accurately represent the information unique to the target domain.

[0185] Optionally, in the image classification apparatus provided in this application embodiment, during the process of processing the M images through the target classification model to obtain target domain feature information corresponding to the target domain, the acquisition module is specifically used to: process the M images through the target classification model to obtain N target feature vectors, where N is an integer and N≥1; and use the N target feature vectors as target domain feature information corresponding to the target domain. In this way, one or more target feature vectors can be flexibly obtained as target domain feature information.

[0186] In the image classification apparatus provided in this application embodiment, the target classification model may include: N target convolutional layers and a second prediction layer for classification prediction, wherein the Nth target convolutional layer is connected to the second prediction layer;

[0187] In the process of processing the M images through the target classification model to obtain N target feature vectors, the processing module is specifically used to: process the M target inputs corresponding to the M images through the i-th target convolutional layer in the target classification model to obtain the i-th target feature vector among the N target feature vectors; where i is an integer and N≥i≥1.

[0188] In this way, one target convolutional layer in the target classification model can output one target feature vector, and thus N target convolutional layers in the target classification model can output N target feature vectors.

[0189] Specifically, in the process of processing the M target inputs corresponding to the M images through the i-th target convolutional layer in the target classification model to obtain the i-th target feature vector among the N target feature vectors, the processing module can be used to:

[0190] The i-th target convolutional layer processes the M target inputs corresponding to the M images respectively to extract M domain feature vectors, wherein one of the M domain feature vectors corresponds to one of the M target inputs;

[0191] Calculate the average value of the M domain feature vectors, and use the average value of the M domain feature vectors as the i-th target feature vector among the N target feature vectors, where M≥2.

[0192] In this way, since each of the N target feature vectors is obtained by averaging the M domain feature vectors, the stability is good and it can accurately represent the information unique to the target domain.

[0193] Optionally, the image classification apparatus provided in this application embodiment further includes a target classification model training module. The target classification model training module can be used for;

[0194] Obtain K training datasets, where each of the K training datasets corresponds to an image category, and the image categories corresponding to the K training datasets are different; each training dataset includes multiple images belonging to different domains, where K is an integer and K≥2;

[0195] Obtain the initial domain classification model;

[0196] The initial domain classification model is trained using the K training datasets to obtain the target classification model.

[0197] In this way, since multiple images in a training dataset belong to different domains but have the same image category, using this kind of training dataset to train the initial domain classification model can treat the same domain as a category for domain classification training. Furthermore, since the images in a training dataset have the same image category, the training can be free from interference from image category and can learn the differences between domains, so that the target classification model can be used to extract the differences between domains (i.e., domain feature information) in the future.

[0198] Optionally, in the process of adjusting the first image classification model based on the target domain feature information to obtain the second image classification model, the processing module can be used to: use the adjusted image classification model as the second image classification model; or, use image samples belonging to the target domain to train the adjusted image classification model to obtain the second image classification model.

[0199] In this way, the adjusted image classification model can be used as the second image classification model. This reduces the difficulty of obtaining the second image classification model, as it is applicable to the classification of images belonging to the target domain. Alternatively, a small number of image samples belonging to the target domain can be used as training samples to train the adjusted image classification model, thus obtaining the second image classification model and further improving its accuracy in recognizing target images.

[0200] The image classification device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0201] The image classification device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0202] The image classification device provided in this application embodiment can achieve... Figures 2 to 9 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0203] Optionally, such as Figure 11As shown, this application embodiment also provides an electronic device 1100, including a processor 1101 and a memory 1102. The memory 1102 stores a program or instructions that can run on the processor 1101. When the program or instructions are executed by the processor 1101, they implement the various steps in any of the image classification methods provided in this application embodiment and achieve the same technical effect. For example, when the program or instructions are executed by the processor 1101, they implement the following process: acquiring a target image, wherein the target image is an image to be classified belonging to a target domain; inputting the target image into a second image classification model; wherein the second image classification model is an image classification model obtained by adjusting a first image classification model based on target domain feature information corresponding to the target domain; the first image classification model is a pre-trained image classification model and is used to classify images belonging to the source domain; and outputting the classification result of the target image. In this way, since the second image classification model is obtained by adjusting the first image classification model based on the target domain feature information corresponding to the target domain, the second image classification model can be applied to the classification of images belonging to the target domain. Compared with related technologies, there is no need to input the image to be classified in the target domain into the domain transfer model for preprocessing. The image to be classified in the target domain is directly classified by the second image classification model, and the image classification speed is faster, which solves the problem of slow image classification speed in related technologies.

[0204] It should be noted that the electronic devices in the embodiments of this application include mobile electronic devices and non-mobile electronic devices.

[0205] Figure 12 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0206] The electronic device 1200 includes, but is not limited to, components such as: radio frequency unit 1201, network module 1202, audio output unit 1203, input unit 1204, sensor 1205, display unit 1206, user input unit 1207, interface unit 1208, memory 1209, and processor 1210.

[0207] Those skilled in the art will understand that the electronic device 1200 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1210 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 12 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0208] The user input unit 1207 is used to acquire a target image, which is a classifying image belonging to the target domain.

[0209] The input unit 1204 is used to input the target image into the second image classification model; wherein, the second image classification model is an image classification model obtained by adjusting the first image classification model based on the target domain feature information corresponding to the target domain; the first image classification model is a pre-trained image classification model and is used to classify images belonging to the source domain;

[0210] The processor 1210 is also used to output the classification result of the target image.

[0211] In the electronic device provided in this application embodiment, a user input unit is used to acquire a target image, which is an image to be classified belonging to a target domain; the input unit is used to input the target image into a second image classification model; wherein, the second image classification model is an image classification model obtained by adjusting a first image classification model based on target domain feature information corresponding to the target domain; the first image classification model is a pre-trained image classification model and is used to classify images belonging to the source domain; the processor is also used to output the classification result of the target image. Thus, since the second image classification model is an image classification model obtained by adjusting a first image classification model based on target domain feature information corresponding to the target domain, the second image classification model can be applied to the classification of images belonging to the target domain. Compared with related technologies, there is no need to preprocess the image to be classified in the target domain by inputting it into a domain transfer model; the image to be classified in the target domain is directly classified through the second image classification model, resulting in faster image classification speed and solving the problem of slow image classification speed in related technologies.

[0212] It should be understood that, in this embodiment, the input unit 1204 may include a graphics processing unit (GPU) 12041 and a microphone 12042. The GPU 12041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1206 may include a display panel 12061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1207 includes a touch panel 12071 and at least one of other input devices 12072. The touch panel 12071 is also called a touch screen. The touch panel 12071 may include a touch detection device and a touch controller. Other input devices 12072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0213] The memory 1209 can be used to store software programs and various data. The memory 1209 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1209 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1209 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0214] Processor 1210 may include one or more processing units; optionally, processor 1210 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1210.

[0215] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0216] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0217] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0218] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0219] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, it will not be described again here.

[0220] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0221] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0222] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image classification method, characterized in that, include: Obtain the first image classification model; The first image classification model is a pre-trained image classification model, used to classify images belonging to the source domain; Obtain target domain feature information corresponding to the target domain. The target domain feature information is used to characterize the features unique to the target domain. The target domain feature information includes N target feature vectors, where N is an integer and N≥1. Obtain source domain feature information corresponding to the source domain in the first image classification model. The source domain feature information is used to characterize features unique to the source domain. The source domain feature information includes N specified feature vectors. The N specified feature vectors in the first image classification model are replaced with the N target feature vectors to obtain the adjusted image classification model; Based on the adjusted image classification model, a second image classification model is obtained, wherein the second image classification model is suitable for classifying images belonging to the target domain; Acquire a target image, wherein the target image is an image to be classified that belongs to the target domain; The target image is input into the second image classification model; Output the classification result of the target image.

2. The method according to claim 1, characterized in that, The first image classification model includes: N specified convolutional layers and a first prediction layer for classification prediction, wherein the Nth specified convolutional layer is connected to the first prediction layer; the i-th specified feature vector among the N specified feature vectors corresponds to the i-th specified convolutional layer among the N specified convolutional layers, where i is an integer and N≥i≥1; The step of obtaining target domain feature information corresponding to the target domain includes: using a target classification model to obtain target domain feature information corresponding to the target domain, wherein the target classification model is a pre-trained domain classification model; The target classification model includes: N target convolutional layers and a second prediction layer for classification prediction; the Nth target convolutional layer is connected to the second prediction layer; the i-th target convolutional layer corresponds to the i-th specified convolutional layer among the N specified convolutional layers; the i-th target feature vector among the N target feature vectors is output by the i-th target convolutional layer. The step of replacing the N specified feature vectors in the first image classification model with the N target feature vectors includes: For each of the N specified feature vectors, perform the following replacement operation: replace the i-th specified feature vector among the N specified feature vectors with the i-th target feature vector among the N target feature vectors; where 1≤i≤N; In the case of N=1, the sum of the output of the first specified convolutional layer and the first target feature vector is used as the input of the first prediction layer; When N≥2 and N-1≥q≥1, the output of the q-th specified convolutional layer and the sum of the q-th target feature vector are used as the input of the (q+1)-th specified convolutional layer, where q is an integer; When N≥2 and q=N, the output of the q-th specified convolutional layer and the sum of the q-th target feature vector are used as the input of the first prediction layer.

3. The method according to claim 1, characterized in that, The acquisition of target domain feature information corresponding to the target domain includes: Obtain the target classification model, which is a pre-trained domain classification model; Obtain M images belonging to the target domain; The M images are input into the target classification model; The M images are processed by the target classification model to obtain target domain feature information corresponding to the target domain; where M is a positive integer.

4. The method according to claim 3, characterized in that, The step of processing the M images using the target classification model to obtain target domain feature information corresponding to the target domain includes: The M images are processed by the target classification model to obtain N target feature vectors, where N is an integer and N≥1; The N target feature vectors are used as target domain feature information corresponding to the target domain.

5. The method according to claim 4, characterized in that, The target classification model includes: N target convolutional layers and a second prediction layer for classification prediction, wherein the Nth target convolutional layer is connected to the second prediction layer; The process of processing the M images using the target classification model to obtain N target feature vectors includes: The M target inputs corresponding to the M images are processed by the i-th target convolutional layer in the target classification model to obtain the i-th target feature vector among the N target feature vectors; where i is an integer and N≥i≥1.

6. The method according to claim 5, characterized in that, The step of processing the M target inputs corresponding to the M images through the i-th target convolutional layer in the target classification model to obtain the i-th target feature vector among the N target feature vectors includes: The i-th target convolutional layer processes the M target inputs corresponding to the M images respectively to extract M domain feature vectors, wherein one of the M domain feature vectors corresponds to one of the M target inputs; Calculate the average value of the M domain feature vectors, and use the average value of the M domain feature vectors as the i-th target feature vector among the N target feature vectors, where M≥2.

7. The method according to claim 3, characterized in that, The training process of the target classification model includes: Obtain K training datasets, where each of the K training datasets corresponds to an image category, and the image categories corresponding to the K training datasets are different; each training dataset includes multiple images belonging to different domains, where K is an integer and K≥2; Obtain the initial domain classification model; The initial domain classification model is trained using the K training datasets to obtain the target classification model.

8. The method according to claim 1, characterized in that, The process of obtaining the second image classification model based on the adjusted image classification model includes: The adjusted image classification model is used as the second image classification model; or, The adjusted image classification model is trained using image samples belonging to the target domain to obtain the second image classification model.

9. An image classification device, characterized in that, include: The module includes an acquisition module, a processing module, an input module, and an output module. The acquisition module is used to: acquire a first image classification model; the first image classification model is a pre-trained image classification model, and is used to classify images belonging to the source domain; acquire target domain feature information corresponding to the target domain, the target domain feature information being used to characterize features unique to the target domain; the target domain feature information includes N target feature vectors, where N is an integer and N≥1; The processing module is configured to acquire source domain feature information corresponding to the source domain in the first image classification model, wherein the source domain feature information is used to characterize features unique to the source domain; the source domain feature information includes N specified feature vectors; replace the N specified feature vectors in the first image classification model with the N target feature vectors to obtain an adjusted image classification model; and based on the adjusted image classification model, obtain a second image classification model, wherein the second image classification model is suitable for classifying images belonging to the target domain; The acquisition module is used to acquire a target image, wherein the target image is an image to be classified that belongs to the target domain; The input module is used to input the target image into the second image classification model; The output module is used to output the classification result of the target image.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that, when executed by the processor, implement the steps of the method as described in any one of claims 1-8.

11. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-8.

Images