Image classification method, device, electronic device and storage medium

Abstract

The present disclosure provides an image classification method, device, electronic device and storage medium, the image classification method comprising: inputting an image to be classified into a trained neural network for feature extraction; obtaining a target classification center feature corresponding to the image to be classified based on the extracted image features and the original classification center feature; wherein the original classification center feature is obtained by training and learning the neural network based on a sample image; based on the target classification center feature, the image to be classified is classified to obtain a classification result. The embodiment of the present application enables a single detection model to adaptively detect images tampered with by multiple tampering methods, thereby improving the applicability of the detection model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to an image classification method, device, electronic device and storage medium. Background Art

[0002] With the continuous advancement of deep fake tampering technology, it has brought great challenges to deep fake detection methods. For example, based on Generative Adversarial Networks (GAN), it is possible to generate highly realistic fake data, thus achieving a fake-to-real effect.

[0003] As the number of forgery methods increases, it is increasingly difficult to use a single detection model to identify multiple types of forged images. In addition, the uneven level of tampering methods will also lead to large differences within the tampered category data. Therefore, it is particularly important to use a single detection model to adaptively detect forged images with multiple tampering methods. Summary of the invention

[0004] The embodiments of the present disclosure at least provide an image classification method, device, electronic device and storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides an image classification method, comprising:

[0006] Input the image to be classified into the trained neural network for feature extraction;

[0007] Based on the extracted image features and the original classification center features, a target classification center feature corresponding to the image to be classified is obtained; wherein the original classification center feature is obtained by training and learning the neural network based on the sample image;

[0008] Based on the target classification center feature, the image to be classified is classified to obtain a classification result.

[0009] In the disclosed embodiment, after the image features of the image to be classified are extracted, the original classification center features are offset according to the extracted image features to obtain the target classification center features, so that the adjusted target classification center features are more suitable for the classification of the current image to be classified. That is, since the offset of the current image to be classified to the original classification center features is generated according to the features of the current image to be classified itself, adaptive adjustment of the classification center features is achieved, which satisfies various types of images to be classified and improves the accuracy of image classification.

[0010] According to the first aspect, in a possible implementation manner, obtaining a target classification center feature corresponding to the image to be classified based on the extracted image features and the original classification center feature includes:

[0011] Based on the extracted image features, obtaining an offset of the target classification center feature relative to the original classification center feature;

[0012] The target classification center feature is obtained based on the offset and the original classification center feature.

[0013] In the disclosed embodiment, the offset of the target classification center feature relative to the original classification center feature is first obtained based on the extracted image features, and then the target classification center feature is obtained based on the offset and the original classification center feature. In this way, not only the determination of the target classification center feature is achieved, but also the determination speed of the target classification center feature is improved.

[0014] According to the first aspect, in a possible implementation, the original classification center feature includes the original real sample class center feature and the original tampered sample class center feature; the target classification center feature includes the target real sample class center feature and the target tampered sample class center feature;

[0015] The step of classifying the image to be classified based on the target classification center feature to obtain a classification result includes:

[0016] Based on the central features of the target real sample class and the central features of the target tampered sample class, the image to be classified is classified to obtain the classification result.

[0017] In the disclosed embodiment, since the original classification center includes the original real sample class center feature and the original tampered sample class center feature, classification is performed according to the two classification center features at the same time when classifying the image to be classified, thereby further improving the accuracy of the test.

[0018] According to the first aspect, in a possible implementation manner, obtaining a target classification center feature corresponding to the image to be classified based on the extracted image features and the original classification center feature includes:

[0019] Based on the extracted image features, respectively obtaining a first offset of the target real sample class center feature relative to the original real sample classification center feature, and obtaining a second offset of the target tampered sample class center feature relative to the original tampered sample class center feature;

[0020] The first offset is added to the original real sample class center feature to obtain the target real sample class center feature, and the second offset is added to the original tampered sample class center feature to obtain the target tampered sample class center feature.

[0021] In the disclosed embodiment, a first offset relative to the original true sample class center and a second offset relative to the original tampered sample class center are predicted simultaneously, and element-by-element addition is performed in the feature dimension to obtain an adjusted classification center, thereby improving the adaptability of the obtained target true sample class center features and target tampered sample class center features.

[0022] According to the first aspect, in a possible implementation manner, classifying the image to be classified based on the target classification center feature to obtain a classification result includes:

[0023] Comparing the extracted image features with the target real sample class center features and the target tampered sample class center features, respectively, to obtain a first difference comparison result between the image to be classified and the target real sample class center features, and a second difference comparison result between the image to be classified and the target tampered sample class center features, respectively;

[0024] Based on the first difference comparison result and the second difference comparison result, the image to be classified is classified to obtain a classification result.

[0025] According to the first aspect, in a possible implementation manner, classifying the image to be classified based on the first difference comparison result and the second difference comparison result to obtain a classification result includes:

[0026] The classification corresponding to the difference comparison result indicating the smallest difference between the first difference comparison result and the second difference comparison result is determined as the classification result corresponding to the image to be classified.

[0027] According to the first aspect, in a possible implementation manner, the image to be classified includes a test image, and the method further includes:

[0028] Obtaining a labeling result corresponding to the test image;

[0029] According to the obtained classification result and the labeling result of the test image, the parameters of the trained neural network are adjusted to obtain a tested neural network.

[0030] In the disclosed embodiment, after obtaining the classification result, the parameters of the neural network are adjusted according to the obtained classification result, so that the performance of the neural network can be further improved, thereby improving the reliability of subsequent image detection.

[0031] In a second aspect, an embodiment of the present disclosure provides an image classification device, including:

[0032] An extraction module is used to input the image to be classified into a trained neural network for feature extraction;

[0033] A calculation module, used to obtain a target classification center feature corresponding to the image to be classified based on the extracted image features and the original classification center feature; wherein the original classification center feature is obtained by training and learning the neural network based on the sample image;

[0034] The classification module is used to classify the image to be classified based on the target classification center feature to obtain a classification result.

[0035] According to the second aspect, in a possible implementation manner, the calculation module is specifically used to:

[0036] Based on the extracted image features, obtaining an offset of the target classification center feature relative to the original classification center feature;

[0037] The target classification center feature is obtained based on the offset and the original classification center feature.

[0038] According to the second aspect, in a possible implementation, the original classification center feature includes the original real sample class center feature and the original tampered sample class center feature; the target classification center feature includes the target real sample class center feature and the target tampered sample class center feature;

[0039] The classification module is specifically used for:

[0040] Based on the central features of the target real sample class and the central features of the target tampered sample class, the image to be classified is classified to obtain the classification result.

[0041] According to the second aspect, in a possible implementation manner, the calculation module is specifically used to:

[0042] Based on the extracted image features, respectively obtaining a first offset of the target real sample class center feature relative to the original real sample classification center feature, and obtaining a second offset of the target tampered sample class center feature relative to the original tampered sample class center feature;

[0043] The first offset is added to the original real sample class center feature to obtain the target real sample class center feature, and the second offset is added to the original tampered sample class center feature to obtain the target tampered sample class center feature.

[0044] According to the second aspect, in a possible implementation manner, the classification module is specifically used to:

[0045] Comparing the extracted image features with the target real sample class center features and the target tampered sample class center features, respectively, to obtain a first difference comparison result between the image to be classified and the target real sample class center features, and a second difference comparison result between the image to be classified and the target tampered sample class center features, respectively;

[0046] Based on the first difference comparison result and the second difference comparison result, the image to be classified is classified to obtain a classification result.

[0047] According to the second aspect, in a possible implementation manner, the classification module is specifically used to:

[0048] The classification corresponding to the difference comparison result indicating the smallest difference between the first difference comparison result and the second difference comparison result is determined as the classification result corresponding to the image to be classified.

[0049] According to the second aspect, in a possible implementation manner, the image to be classified includes a test image; and the device further includes:

[0050] An acquisition module, used to acquire the annotation result corresponding to the test image;

[0051] The adjustment module is used to adjust the parameters of the trained neural network according to the classification result and the labeling result of the test image to obtain a tested neural network.

[0052] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the image classification method described in the first aspect is performed.

[0053] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the image classification method as described in the first aspect is executed.

[0054] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0056] Figure 1 A flowchart of an image classification method provided by an embodiment of the present disclosure is shown;

[0057] Figure 2 A flowchart diagram showing another image classification method provided by an embodiment of the present disclosure is shown;

[0058] Figure 3 A flow chart of a method for determining a target classification center provided by an embodiment of the present disclosure is shown;

[0059] Figure 4a A schematic diagram of a process of classifying an image to be classified according to original classification center features provided by an embodiment of the present disclosure is shown;

[0060] Figure 4b A schematic diagram of a process for determining a target classification center feature provided by an embodiment of the present disclosure is shown;

[0061] Figure 4c Another schematic diagram of a process for determining a target classification center feature provided by an embodiment of the present disclosure is shown;

[0062] Figure 5 A flow chart of a method for classifying an image to be classified to obtain a classification result provided by an embodiment of the present disclosure is shown;

[0063] Figure 6 A schematic diagram of the structure of a neural network provided by an embodiment of the present disclosure is shown;

[0064] Figure 7 Another method flow chart of classifying an image to be classified to obtain a classification result provided by an embodiment of the present disclosure is shown;

[0065] Figure 8 A flowchart diagram of another image classification method provided by an embodiment of the present disclosure is shown;

[0066] Fig. 9 A schematic diagram of the structure of an image classification device provided by an embodiment of the present disclosure is shown;

[0067] Fig.10 A schematic diagram showing the structure of another image classification device provided by an embodiment of the present disclosure is shown;

[0068] Fig.11 A schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0070] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0071] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0072] Deep Fake technology is also known as artificial intelligence face-changing technology in the industry. This technology can digitally process images or videos to imitate a specific person, generate a large number of false audio and video content, and make it difficult to distinguish the authenticity of information. Due to the potential social harm of deep fake content, it is very necessary to detect and authenticate fake content.

[0073] Existing methods of tampering and forgery include changing faces, changing expressions, changing facial features, changing facial attributes, and full-image simulation generation. The tampered images generated by various tampering methods vary greatly, resulting in large differences within the tampered data class, which brings difficulties to model detection.

[0074] In addition, when the tampering method is unknown, it is difficult to achieve stable results in a variety of tampered data using a single detection model. Therefore, it is particularly important to use a single detection model to adaptively detect forged images with multiple tampering methods.

[0075] Based on the above research, the present disclosure provides an image classification method, first inputting the image to be classified into a trained neural network for feature extraction; then based on the extracted image features and the original classification center features, obtaining the target classification center features corresponding to the image to be classified; wherein the original classification center features are obtained by the neural network through training and learning based on sample images; then based on the target classification center features, the image to be classified is classified to obtain a classification result. In this way, for different images to be classified, corresponding target classification center features will be obtained, and then the images to be classified will be classified based on the target classification center features, so that the current model can be adapted to different types of images to be classified.

[0076] To facilitate understanding of this embodiment, an image classification method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the image classification method provided in the embodiment of the present disclosure is generally an electronic device with certain computing capabilities, and the electronic device includes, for example: a terminal device or a server or other processing device, and the terminal device can be a mobile device, a user terminal, a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the image classification method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0077] See also Figure 1 FIG. 1 is a flowchart of an image classification method provided by an embodiment of the present disclosure. The image classification method includes the following S101 to S103:

[0078] S101, input the image to be classified into the trained neural network for feature extraction.

[0079] Exemplarily, the image to be classified refers to an image that needs to be detected for authenticity, which may be an image or a frame of image obtained after decoding the target video, which is not limited here.

[0080] It should be understood that a trained neural network can be obtained by inputting a large number of sample images with labels into the neural network for training and learning. The large number of sample images may include a number of real sample images and a number of tampered sample images. In this way, the original classification center features for classifying the images to be classified can be obtained after training a large number of sample images.

[0081] S102, based on the extracted image features and the original classification center features, obtain the target classification center features corresponding to the image to be classified; wherein the original classification center features are obtained by training and learning the neural network based on sample images.

[0082] For example, based on the extracted image features, the offset of the target classification center feature relative to the original classification center feature can be obtained, and then the offset is added to the original classification center feature to obtain the new classification center feature. In this way, the obtained target can be adapted to the classification of the current image to be classified.

[0083] S103, classifying the image to be classified based on the target classification center feature to obtain a classification result.

[0084] For example, the extracted image features of the image to be classified can be compared with the target classification center features, and if the comparison result is within a preset range, it can be determined that the image to be classified is a real image or a tampered image. The specific comparison and classification method will be described in detail later.

[0085] In the disclosed embodiment, after the image features of the image to be classified are extracted, the original classification center features are offset according to the extracted image features to obtain the target classification center features, so that the adjusted target classification center features are more suitable for the classification of the current image to be classified. That is, since the offset of the current image to be classified to the original classification center features is generated according to the features of the current image to be classified itself, adaptive adjustment of the classification center features is achieved, which satisfies various types of images to be classified and improves the accuracy of image classification.

[0086] In some embodiments, in order to further improve the accuracy of image detection, the original classification center features include the original real sample class center features and the original tampered sample class center features. That is, the technical solution provided by the present disclosure can be implemented by a neural network, and in the process of training the neural network, it will be trained according to the authenticity of the input sample image to obtain the original real sample class center features and the original tampered sample class center features. Therefore, see Figure 2 FIG. 2 is a flowchart of another image classification method provided by an embodiment of the present disclosure. The image classification method includes the following S201 to S203:

[0087] S201, input the image to be classified into the trained neural network for feature extraction.

[0088] Among them, step S201 is similar to the above step S101 and will not be repeated here.

[0089] S202, based on the extracted image features and the original real sample class center features, obtain the target real sample class center features corresponding to the image to be classified, and based on the extracted image features and the original tampered sample class center features, obtain the target tampered sample class center features corresponding to the image to be classified.

[0090] For example, Figure 3 As shown, step S202 may include the following S2021-2022:

[0091] S2021, based on the extracted image features, respectively obtain a first offset of the target real sample class center feature relative to the original real sample classification center feature, and obtain a second offset of the target tampered sample class center feature relative to the original tampered sample class center feature.

[0092] Exemplarily, the first offset and the second offset of the target tampered sample class center feature relative to the original tampered sample class center feature can be obtained according to the following formula and the currently extracted image features.

[0093]

[0094] Where x is the extracted image feature; [WIwN;] is the classification center feature; τ is the scale factor; b y Corresponding to different classification center features.

[0095] Specifically, in a neural network, the offset can be obtained through two fully connected layers of the neural network. The two connection layers have a normalization processing function (SoftMax) and a linear rectifier function (Rectified Linear Unit, ReLU) respectively. The ReLU, also known as the rectified linear unit, is a commonly used activation function in artificial neural networks, usually referring to nonlinear functions represented by ramp functions and their variants.

[0096] S2022, adding the first offset to the original real sample class center feature to obtain the target real sample class center feature, and adding the second offset to the original tampered sample class center feature to obtain the target tampered sample class center feature.

[0097] For example, the image to be classified is subjected to feature extraction by a trained neural network to obtain image features, which are high-dimensional features. After dimensionality reduction, a schematic diagram of a two-dimensional plane is formed as shown in FIG. Figure 4a As shown in, where A refers to w r Represents the original real sample classification center feature, B refers to w frepresents the classification center feature of the original tampered sample class, “+” represents the image to be classified whose classification result is a tampered image, and “·” represents the image to be classified whose classification result is a real image.

[0098] Exemplarily, the t-SNE algorithm can be used to map high-dimensional features to a two-dimensional plane. Among them, t-SNE is a machine learning algorithm for dimensionality reduction. Specifically, t-SNE is a nonlinear dimensionality reduction algorithm suitable for reducing high-dimensional data to 2D or 3D for visualization.

[0099] S203, classifying the image to be classified based on the central feature of the target real sample class and the central feature of the target tampered sample class to obtain the classification result.

[0100] For example, see Figure 4a As shown, the original real sample class classification center feature A and the original tampered sample class classification center feature B determine the original classification surface S. If the image to be classified is classified according to the original real sample class classification center feature A and the original tampered sample class classification center feature B, the image to be classified X, which was originally a real image, will be classified as a tampered image, and the image to be classified Y, which was originally a tampered image, will be classified as a real image, which will lead to inaccurate classification of the image to be classified X and the image to be classified Y.

[0101] However, after following the steps of this protocol, see Figure 4b As shown, firstly, according to the image features of the image to be classified X extracted, the first offset and the second offset are determined respectively, and then based on the first offset and the second offset, the target real sample class classification center feature A1 and the target tampered sample class classification center feature B1 are obtained. The target real sample class classification center feature A1 and the target tampered sample class classification center feature B1 re-determine the target classification surface S1, from Figure 4b It can be seen that after adjusting the original real sample class classification center feature A and the original tampered sample class classification center feature B, the classification result of the image to be classified X is the real image, that is, the classification result at this time is accurate.

[0102] Similarly, see Figure 4c As shown, firstly, according to the image features of the image to be classified Y extracted, the first offset and the second offset are determined respectively, and then based on the first offset and the second offset, the target real sample class classification center feature A2 and the target tampered sample class classification center feature B2 are obtained. The target real sample class classification center feature A2 and the target tampered sample class classification center feature B2 redefine the target classification surface S2, from Figure 4cIt can be seen that after adjusting the original real sample class classification center feature A and the original tampered sample class classification center feature B, the classification result of the image to be classified Y is a tampered image, that is, the classification result at this time is accurate, which improves the applicability of the neural network.

[0103] For example, Figure 5 As shown, in some implementation modes, the above step S203 may include the following S2031 to S2032:

[0104] S2031, respectively comparing the extracted image features with the target real sample class central features and the target tampered sample class central features, to obtain a first difference comparison result between the image to be classified and the target real sample class central features, and a second difference comparison result between the image to be classified and the target tampered sample class central features.

[0105] S2032: Classify the image to be classified based on the first difference comparison result and the second difference comparison result to obtain a classification result.

[0106] For example, see Figure 6 As shown, the image to be classified (a) is input into the neural network for feature extraction, and the central features of the target real sample class and the central features of the target tampered sample class can be obtained respectively according to the extracted image features; then the image to be classified is compared with the central features of the target real sample class and the central features of the target tampered sample class respectively to obtain the first comparison result and the second comparison result; and then the classification result of the current image to be classified is determined according to the comparison results.

[0107] For example, if the comparison difference between the central feature of the current image to be classified and the target real sample class is smaller, it means that the probability that the current image to be classified is a real image is greater; conversely, if the comparison difference between the central feature of the current image to be classified and the target tampered sample class is smaller, it means that the probability that the current image to be classified is a tampered image is greater. Therefore, in some embodiments, the classification corresponding to the difference comparison result indicating the smallest difference between the first difference comparison result and the second difference comparison result can be determined as the classification result corresponding to the image to be classified.

[0108] In some other embodiments, see Figure 7 As shown, for the above S203, when the image to be classified is classified based on the target real sample class center feature and the target tampered sample class center feature to obtain the classification result, the following S203a-203b may be included:

[0109] S203a, respectively calculating the cosine similarity between the extracted image feature and the target real sample class central feature and the target tampered sample class central feature, and respectively obtaining a first calculation result between the image to be classified and the real sample class central feature, and a second calculation result between the image to be classified and the target tampered sample class central feature.

[0110] Among them, cosine similarity, also known as cosine similarity, evaluates the similarity of two vectors by calculating the cosine value of the angle between them. Cosine similarity plots vectors into vector space according to their coordinate values, such as the most common two-dimensional space.

[0111] S203b determines the category corresponding to the larger calculation result between the first calculation result and the second calculation result as the classification result corresponding to the image to be classified.

[0112] Among them, cosine similarity measures the similarity between two vectors by measuring the cosine value of the angle between them. The cosine value of a 0-degree angle is 1, while the cosine value of any other angle is not greater than 1; and its minimum value is -1. Therefore, the cosine value of the angle between two vectors determines whether the two vectors point roughly in the same direction. When two vectors have the same direction, the cosine similarity value is 1; when the angle between the two vectors is 90°, the cosine similarity value is 0; when the two vectors point in completely opposite directions, the cosine similarity value is -1. This result is independent of the length of the vector, but only related to the direction in which the vector points. Cosine similarity is usually used in positive space, so the value given is between -1 and 1.

[0113] In this embodiment, the larger the settlement result is, the greater the similarity between the image feature and the classification center feature is (the smaller the difference is), therefore, the classification corresponding to the larger calculation result of the first calculation result and the second calculation result is determined as the classification result corresponding to the image to be classified. For example, if the first calculation result is larger, the current image to be classified is determined to be a real image; and if the second calculation result is larger, the current image to be classified is determined to be a tampered image.

[0114] In some implementations, the image to be classified includes a test image; see Figure 8 FIG. 1 is a flowchart of another image classification method provided by an embodiment of the present disclosure. Figure 1 The method in is different from that in that it also includes the following steps S104-S105:

[0115] S104: Obtain a labeling result corresponding to the test image.

[0116] S105, adjusting parameters of the trained neural network according to the obtained classification result and the labeling result of the test image to obtain a tested neural network.

[0117] Please refer again Figure 6 It can be understood that when the image to be classified is a test image, each test image will have a corresponding annotation result, which can be a real image or a tampered image. In order to enable the neural network to provide detection accuracy while being applicable to images tampered with in different ways, the parameters of the neural network can also be adjusted according to the classification results obtained by the above method and the annotation results of the test image. That is, after classifying the test image and obtaining the classification result, the classification result and the annotation result can be compared. If the annotation result is a real image and the decomposition result is also a real image, it indicates that the current classification result is accurate. Otherwise, it indicates that the current classification result is inaccurate, and the parameters of the neural network need to be further adjusted until the classification result is consistent with the annotation result, thereby improving the performance of the neural network.

[0118] Exemplarily, the loss L may be determined according to the following formula, and the parameters of the neural network may be adjusted based on the loss L.

[0119] L(x, y) = -logP(Y = y|x)

[0120] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0121] Based on the same technical concept, an image classification device corresponding to the image classification method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned image classification method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0122] Reference Fig. 9 FIG. 5 is a schematic diagram of an image classification device 500 provided in an embodiment of the present disclosure, and the image classification device includes:

[0123] Extraction module 501, used for inputting the image to be classified into the trained neural network for feature extraction;

[0124] A calculation module 502 is used to obtain a target classification center feature corresponding to the image to be classified based on the extracted image features and the original classification center feature; wherein the original classification center feature is obtained by training and learning the neural network based on the sample image;

[0125] The classification module 503 is used to classify the image to be classified based on the target classification center feature to obtain a classification result.

[0126] In a possible implementation manner, the calculation module 502 is specifically configured to:

[0127] Based on the extracted image features, obtaining an offset of the target classification center feature relative to the original classification center feature;

[0128] The target classification center feature is obtained based on the offset and the original classification center feature.

[0129] In a possible implementation, the original classification center feature includes the original real sample class center feature and the original tampered sample class center feature; the target classification center feature includes the target real sample class center feature and the target tampered sample class center feature;

[0130] The classification module 503 is specifically used for:

[0131] Based on the central features of the target real sample class and the central features of the target tampered sample class, the image to be classified is classified to obtain the classification result.

[0132] In a possible implementation manner, the calculation module 502 is specifically configured to:

[0133] Based on the extracted image features, respectively obtaining a first offset of the target real sample class center feature relative to the original real sample classification center feature, and obtaining a second offset of the target tampered sample class center feature relative to the original tampered sample class center feature;

[0134] The first offset is added to the original real sample class center feature to obtain the target real sample class center feature, and the second offset is added to the original tampered sample class center feature to obtain the target tampered sample class center feature.

[0135] In a possible implementation manner, the classification module 503 is specifically used to:

[0136] Comparing the extracted image features with the target real sample class center features and the target tampered sample class center features, respectively, to obtain a first difference comparison result between the image to be classified and the target real sample class center features, and a second difference comparison result between the image to be classified and the target tampered sample class center features, respectively;

[0137] Based on the first difference comparison result and the second difference comparison result, the image to be classified is classified to obtain a classification result.

[0138] In a possible implementation manner, the classification module 503 is specifically used to:

[0139] The classification corresponding to the difference comparison result indicating the smallest difference between the first difference comparison result and the second difference comparison result is determined as the classification result corresponding to the image to be classified.

[0140] In one possible implementation, see Fig.10 As shown, the image to be classified includes a test image; the device also includes:

[0141] An acquisition module 504 is used to acquire a labeling result corresponding to the test image;

[0142] The adjustment module 505 is used to adjust the parameters of the trained neural network according to the obtained classification result and the labeling result of the test image to obtain a tested neural network.

[0143] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0144] Based on the same technical concept, the embodiment of the present disclosure also provides an electronic device. Fig.11 , which is a schematic diagram of the structure of an electronic device 700 provided in an embodiment of the present disclosure, including a processor 701, a memory 702, and a bus 703. The memory 702 is used to store execution instructions, including a memory 7021 and an external memory 7022; the memory 7021 is also called an internal memory, which is used to temporarily store the operation data in the processor 701 and the data exchanged with the external memory 7022 such as a hard disk. The processor 701 exchanges data with the external memory 7022 through the memory 7021.

[0145] In the embodiment of the present application, the memory 702 is specifically used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 701. That is, when the electronic device 700 is running, the processor 701 communicates with the memory 702 through the bus 703, so that the processor 701 executes the application code stored in the memory 702, and then executes the method described in any of the above embodiments.

[0146] Among them, the memory 702 can be, but is not limited to, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0147] The processor 701 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0148] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 700. In other embodiments of the present application, the electronic device 700 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0149] The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the image classification method in the above method embodiment are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0150] The present disclosure also provides a computer program product that carries a program code. The program code includes instructions that can be used to execute the steps of the image classification method in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0151] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0152] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0153] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0154] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0155] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0156] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. An image classification method, characterized in that: include: Input the image to be classified into the trained neural network for feature extraction; Based on the fully connected layer of the neural network, an offset prediction is performed on the extracted image features to obtain an offset of the target classification center feature relative to the original classification center feature, and the offset is added to the original classification center feature to obtain the target classification center feature; wherein the original classification center feature is obtained by training and learning the neural network based on the sample image; the original classification center feature includes the original real sample class center feature and the original tampered sample class center feature; the target classification center feature includes the target real sample class center feature and the target tampered sample class center feature; Based on the central features of the target real sample class and the central features of the target tampered sample class, the image to be classified is classified to obtain a classification result; The fully connected layer based on the neural network predicts the offset of the extracted image features to obtain the offset of the target classification center feature relative to the original classification center feature, and adds the offset to the original classification center feature to obtain the target classification center feature, including: Based on the fully connected layer of the neural network, the extracted image features are predicted by offset, and a first offset of the target real sample class center feature relative to the original real sample classification center feature is obtained, and a second offset of the target tampered sample class center feature relative to the original tampered sample class center feature is obtained; Adding the first offset to the original real sample class center feature to obtain the target real sample class center feature, and adding the second offset to the original tampered sample class center feature to obtain the target tampered sample class center feature; The offset is obtained by the following formula: Among them, x is the extracted image feature; τ is the scale factor; b y is the offset for predicting the classification center feature with label y, corresponding to different classification center features; j is the traversal subscript, [w1,...,w N ] is the classification center feature.

2. The method according to claim 1, characterized in that The classifying the image to be classified based on the target real sample class center feature and the target tampered sample class center feature to obtain a classification result includes: Comparing the extracted image features with the target real sample class center features and the target tampered sample class center features, respectively, to obtain a first difference comparison result between the image to be classified and the target real sample class center features, and a second difference comparison result between the image to be classified and the target tampered sample class center features, respectively; Based on the first difference comparison result and the second difference comparison result, the image to be classified is classified to obtain a classification result.

3. The method according to claim 2, characterized in that The classifying the image to be classified based on the first difference comparison result and the second difference comparison result to obtain a classification result includes: The classification corresponding to the difference comparison result indicating the smallest difference between the first difference comparison result and the second difference comparison result is determined as the classification result corresponding to the image to be classified.

4. The method according to any one of claims 1 to 3, characterized in that The image to be classified includes a test image, and the method further includes: Obtaining a labeling result corresponding to the test image; According to the obtained classification result and the labeling result of the test image, the parameters of the trained neural network are adjusted to obtain a tested neural network.

5. An image classification device, characterized in that: The device comprises: An extraction module is used to input the image to be classified into a trained neural network for feature extraction; A calculation module, used for predicting the offset of the extracted image features based on the fully connected layer of the neural network, obtaining the offset of the target classification center feature relative to the original classification center feature, and adding the offset to the original classification center feature to obtain the target classification center feature; wherein the original classification center feature is obtained by training and learning the neural network based on the sample image; the original classification center feature includes the original real sample class center feature and the original tampered sample class center feature; the target classification center feature includes the target real sample class center feature and the target tampered sample class center feature; A classification module, used to classify the image to be classified based on the central feature of the target real sample class and the central feature of the target tampered sample class to obtain a classification result; The calculation module is specifically used for: Based on the fully connected layer of the neural network, the extracted image features are predicted by offset, and a first offset of the target real sample class center feature relative to the original real sample classification center feature is obtained, and a second offset of the target tampered sample class center feature relative to the original tampered sample class center feature is obtained; Adding the first offset to the original real sample class center feature to obtain the target real sample class center feature, and adding the second offset to the original tampered sample class center feature to obtain the target tampered sample class center feature; The offset is obtained by the following formula: Among them, x is the extracted image feature; τ is the scale factor; b y is the offset for predicting the classification center feature with label y, corresponding to different classification center features; j is the traversal subscript, [w1,...,w N ] is the classification center feature.

6. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the image classification method as described in any one of claims 1 to 4 is performed.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image classification method according to any one of claims 1 to 4 is executed.

Images