Image category labeling method and device, electronic equipment and storage medium

By extracting and automatically labeling the image to be marked, the problem of difficulty in identifying and high labeling costs in the live detection model in new scenarios is solved, and efficient image category annotation and model generalization improvement is achieved.

CN118982697APending Publication Date: 2024-11-19BEIJING ZITIAO NETWORK TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310521434.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-10
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

The existing live detection model cannot be correctly identified in new attack scenarios, and the generalization is reduced due to the overfitting of the model, and a large number of labeled sample images are required for training, resulting in high cost of manual labeling.

Method used

By extracting the image to be marked, obtaining the image label category feature vector in the preset image label category set, determining that the image label category of the image to be marked is the preset image label category closest to it, and realizing automatic labeling of the image category.

Benefits of technology

It improves the fine-grainedness and efficiency of image annotation, realizes the automation of high-fine-grained image category annotation, enhances the generalization of the model, and reduces the cost of manual annotation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982697A_ABST
    Figure CN118982697A_ABST
Patent Text Reader

Abstract

The invention provides an image category labeling method and device, electronic equipment and a storage medium. According to one specific embodiment, the method comprises the steps that feature extraction is carried out on a to-be-labeled image, to-be-labeled image features are obtained, the to-be-labeled image features comprise at least one N-dimensional to-be-labeled feature vector, and N is a positive integer; obtaining an N-dimensional image annotation category feature vector corresponding to each preset image annotation category in a preset image annotation category set; based on the image annotation category corresponding to the to-be-annotated feature vector, the image annotation category of the to-be-annotated image is determined, and the image annotation category corresponding to the to-be-annotated feature vector is a preset image annotation category of which the corresponding image annotation category feature vector in each preset image category is closest to the to-be-annotated feature vector. According to the invention, automation of image category labeling is realized, subsequent training of a corresponding model based on the labeled image is facilitated, and generalization of the model is further improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to an image category labeling method, device, electronic device, and storage medium. Background Art

[0002] With the continuous development of deep learning, image detection is moving towards unsupervised and automated development. Current image detection tasks are often treated as two-classification tasks by deep learning models. For example, in the face liveness detection scenario, the true and false attributes of the face are first characterized on a large amount of data, and then the liveness detection model is trained using labeled images. Finally, the trained liveness detection model is relied upon to distinguish between true and false faces.

[0003] However, current liveness detection models often fail to correctly identify in many new attack scenarios. This is because in the original training set, the model overfits to the feature information about the data, which reduces the generalization of the model. In order to improve the generalization of the model, a large number of labeled sample images are required for model training, and the labeling of general sample images requires a lot of manual costs.

[0004] Therefore, how to realize automatic labeling of image categories is of great significance. Summary of the invention

[0005] The embodiments of the present disclosure provide an image category labeling method, device, electronic device, and storage medium.

[0006] In a first aspect, an embodiment of the present disclosure provides an image category labeling method, the method comprising: performing feature extraction on an image to be labeled to obtain image features to be labeled, wherein the image features to be labeled include at least one N-dimensional feature vector to be labeled, and N is a positive integer; obtaining an N-dimensional image labeling category feature vector corresponding to each preset image labeling category in a preset image labeling category set; determining the image labeling category of the image to be labeled based on the image labeling category corresponding to the feature vector to be labeled, wherein the image labeling category corresponding to the feature vector to be labeled is the preset image labeling category whose image labeling category feature vector corresponding to each of the preset image categories is closest to the feature vector to be labeled.

[0007] In an optional embodiment, the N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the above-mentioned preset image annotation category set is pre-obtained through the following training steps: obtaining a sample image set and an N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the above-mentioned preset image annotation category set; obtaining the model structure and model parameter information of the initial feature extraction model, the initial image generation model and the initial image category discrimination model; based on the above-mentioned sample image set, the adjusted N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the above-mentioned preset image annotation category set, the above-mentioned initial feature extraction model, the above-mentioned initial image generation model and the above-mentioned initial image category discrimination model are trained; and the trained N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the above-mentioned preset image annotation category set is determined as the N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the above-mentioned preset image annotation category set.

[0008] In an optional embodiment, a parameter adjustment operation is performed on the model parameter information of the sample images in the above-mentioned sample image set, the above-mentioned initial feature extraction model, the initial image generation model and the above-mentioned initial image category discrimination model, including: for the sample images in the above-mentioned sample image set, the following parameter adjustment operations are performed until the preset training end condition is met: the sample image is input into the above-mentioned initial feature extraction model to obtain the sample image features of the sample image, wherein the sample image features of the sample image include at least one N-dimensional sample image feature vector; for at least one sample image feature vector in the sample image features of the sample image, based on the initial image annotation category feature vector that is closest to the sample image feature vector in each of the above-mentioned initial image annotation category feature vectors, an image annotation category coding feature vector at a corresponding position in the image annotation category coding feature corresponding to the sample image is generated, wherein the image annotation category coding feature vector is an N-dimensional vector; the image annotation category coding feature corresponding to the sample image is input into the above-mentioned initial image generation model to obtain the image annotation category coding feature corresponding to the sample image Reconstruct an image; input the reconstructed image corresponding to the sample image into the above-mentioned initial image category discrimination model to obtain an image category discrimination result corresponding to the reconstructed image of the sample image, wherein the image category discrimination result is an original image discrimination result or a reconstructed image discrimination result, and the above-mentioned original image discrimination result and the above-mentioned reconstructed image discrimination result are respectively used to indicate that the input image is an original image or a reconstructed image; based on at least one of the first loss function, the second loss function and the third loss function, adjust the N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the above-mentioned preset image annotation category set, the model parameters of the above-mentioned initial feature extraction model, the above-mentioned initial image generation model and the above-mentioned initial image category discrimination model, wherein the above-mentioned first loss function is used to characterize the difference between the sample image feature and the image annotation category coding feature of the above-mentioned sample image, the above-mentioned second loss function is used to characterize the difference between the sample image and the corresponding reconstructed image, and the above-mentioned third loss function is used to characterize the difference between the image category discrimination result corresponding to the sample image and the above-mentioned reconstructed image discrimination result.

[0009] In some optional embodiments, the above-mentioned determining the image annotation category of the image to be annotated based on the image annotation category corresponding to the feature vector to be annotated includes: determining the image annotation category corresponding to each feature vector to be annotated in at least one of the feature vectors to be annotated; merging the image annotation category corresponding to each feature vector to be annotated in at least one of the feature vectors to be annotated; and determining the image annotation categories obtained after the merging as the image annotation category of the image to be annotated.

[0010] In some optional embodiments, the determining the image annotation category of the image to be annotated based on the image annotation category corresponding to the feature vector to be annotated includes: determining the image annotation category corresponding to each feature vector to be annotated in at least one of the feature vectors to be annotated; counting the number of occurrences of each of the preset image annotation categories in the image annotation categories corresponding to each feature vector to be annotated in at least one of the feature vectors to be annotated; and determining the image annotation category of the image to be annotated according to the number of occurrences of each of the preset image annotation categories obtained by counting.

[0011] In some optional embodiments, determining the image annotation category of the image to be annotated based on the statistically obtained number of occurrences of each of the preset image annotation categories includes: determining the preset image annotation category in the preset image annotation category set whose corresponding number of occurrences is greater than a preset number threshold as the image annotation category of the image to be annotated.

[0012] In some optional implementations, determining the image annotation category of the image to be annotated based on the statistically obtained number of occurrences of each of the preset image annotation categories includes: determining the first preset number of preset image annotation categories with the largest number of corresponding occurrences in the preset image annotation category set as the image annotation category of the image to be annotated.

[0013] In some optional implementations, the preset image annotation category set includes image categories in a face liveness detection scenario, and the images to be annotated include images that need to be annotated with image categories in a face liveness detection scenario.

[0014] In a second aspect, an embodiment of the present disclosure provides an image category labeling device, the device comprising: an image acquisition unit, configured to perform feature extraction on an image to be labeled to obtain image features to be labeled, wherein the image features to be labeled include at least one N-dimensional feature vector to be labeled, and the N is a positive integer; a vector acquisition unit, configured to obtain an N-dimensional image labeling category feature vector corresponding to each preset image labeling category in a preset image labeling category set; a vector determination unit, configured to determine the image labeling category of the image to be labeled based on the image labeling category corresponding to the feature vector to be labeled, wherein the image labeling category corresponding to the feature vector to be labeled is the preset image labeling category whose image labeling category feature vector corresponding to each of the preset image categories is closest to the feature vector to be labeled.

[0015] In an optional embodiment, the N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the above-mentioned preset image annotation category set is pre-obtained through the following training steps: obtaining a sample image set and an N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the above-mentioned preset image annotation category set; obtaining the model structure and model parameter information of the initial feature extraction model, the initial image generation model and the initial image category discrimination model; training the trained N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the above-mentioned preset image annotation category set, the above-mentioned initial feature extraction model, the above-mentioned initial image generation model and the above-mentioned initial image category discrimination model based on the above-mentioned sample image set; and determining the adjusted N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the above-mentioned preset image annotation category set as the N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the above-mentioned preset image annotation category set.

[0016] In an optional embodiment, a parameter adjustment operation is performed on the model parameter information of the sample images in the above-mentioned sample image set, the above-mentioned initial feature extraction model, the initial image generation model and the above-mentioned initial image category discrimination model, including: for the sample images in the above-mentioned sample image set, the following parameter adjustment operations are performed until the preset training end condition is met: the sample image is input into the above-mentioned initial feature extraction model to obtain the sample image features of the sample image, wherein the sample image features of the sample image include at least one N-dimensional sample image feature vector; for each sample image feature vector in the sample image features of the sample image, based on the initial image annotation category feature vector in the above-mentioned initial image annotation category feature vector that is closest to the sample image feature vector, an image annotation category coding feature vector at a corresponding position in the image annotation category coding feature corresponding to the sample image is generated, wherein the image annotation category coding feature vector is an N-dimensional vector; the image annotation category coding feature corresponding to the sample image is input into the above-mentioned initial image generation model to obtain the image annotation category coding feature corresponding to the sample image reconstructed image; inputting the reconstructed image corresponding to the sample image into the above-mentioned initial image category discrimination model to obtain an image category discrimination result corresponding to the reconstructed image of the sample image, wherein the image category discrimination result is the original image discrimination result or the reconstructed image discrimination result, and the above-mentioned original image discrimination result and the above-mentioned reconstructed image discrimination result are respectively used to indicate that the input image is the original image or the reconstructed image; based on at least one of the first loss function, the second loss function and the third loss function, adjusting the N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the above-mentioned preset image annotation category set, the model parameters of the above-mentioned initial feature extraction model, the above-mentioned initial image generation model and the above-mentioned initial image category discrimination model, wherein the above-mentioned first loss function is used to characterize the difference between the sample image feature and the image annotation category coding feature of the above-mentioned sample image, the above-mentioned second loss function is the difference between the sample image and the corresponding reconstructed image, and the above-mentioned third loss function is the difference between the image category discrimination result corresponding to the sample image and the above-mentioned reconstructed image discrimination result.

[0017] In some optional embodiments, the above-mentioned determining the image annotation category of the image to be annotated based on the image annotation category corresponding to the feature vector to be annotated includes: determining the image annotation category corresponding to each feature vector to be annotated in at least one of the feature vectors to be annotated; merging the image annotation category corresponding to each feature vector to be annotated in at least one of the feature vectors to be annotated; and determining the image annotation categories obtained after the merging as the image annotation category of the image to be annotated.

[0018] In some optional embodiments, the determining the image annotation category of the image to be annotated based on the image annotation category corresponding to the feature vector to be annotated includes: determining the image annotation category corresponding to each feature vector to be annotated in at least one of the feature vectors to be annotated; counting the number of occurrences of each of the preset image annotation categories in the image annotation categories corresponding to each feature vector to be annotated in at least one of the feature vectors to be annotated; and determining the image annotation category of the image to be annotated according to the number of occurrences of each of the preset image annotation categories obtained by counting.

[0019] In some optional embodiments, determining the image annotation category of the image to be annotated based on the statistically obtained number of occurrences of each of the preset image annotation categories includes: determining the preset image annotation category in the preset image annotation category set whose corresponding number of occurrences is greater than a preset number threshold as the image annotation category of the image to be annotated.

[0020] In some optional implementations, determining the image annotation category of the image to be annotated based on the statistically obtained number of occurrences of each of the preset image annotation categories includes: determining the first preset number of preset image annotation categories with the largest number of corresponding occurrences in the preset image annotation category set as the image annotation category of the image to be annotated.

[0021] In some optional implementations, the preset image annotation category set includes image categories in a face liveness detection scenario, and the images to be annotated include images that need to be annotated with image categories in a face liveness detection scenario.

[0022] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0023] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by one or more processors, implements the method described in any implementation manner in the first aspect.

[0024] In order to achieve automatic labeling of image categories, the image category labeling method, device, electronic device and storage medium provided by the embodiments of the present disclosure extract features of the image to be labeled, obtain the image features to be labeled, obtain the image labeling category feature vector corresponding to each preset image labeling type in the preset image labeling category set, and determine the preset image labeling category whose distance between the feature vector to be labeled included in the image features to be labeled and the image labeling category feature vector is closest, and then determine it as the image labeling category of the image to be labeled. The fine-grainedness of image labeling can be improved. Compared with the binary labeling of traditional detection tasks, the automation of high-grained image category labeling can be achieved, and the efficiency of image labeling category determination can be improved. Subsequently, the distribution characteristics of the existing image set can be comprehensively analyzed through the labeled image, and the image can be targetedly expanded according to the needs of the training task. The model is iteratively updated using the expanded image data, so as to achieve targeted improvement of model capabilities and further improve the generalization of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Other features, objects and advantages of the present disclosure will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are only for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. In the drawings:

[0026] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied;

[0027] Figure 2 is a flowchart of an embodiment of an image category labeling method according to the present disclosure;

[0028] Figure 3 is a flowchart of an embodiment of the image annotation category feature vector training step according to the present disclosure;

[0029] Figure 4 is a schematic diagram of an application scenario of the image category labeling method disclosed in the present invention;

[0030] Figure 5 is a flowchart of another embodiment of the image category labeling method according to the present disclosure;

[0031] Figure 6 is a structural schematic diagram of an embodiment of an image category labeling device according to the present disclosure;

[0032] Figure 7 It is a structural diagram of a computer system of an electronic device suitable for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] The present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0034] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0035] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the image category labeling method, apparatus, electronic device, and storage medium of the present disclosure can be applied.

[0036] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0037] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as natural language processing applications, voice recognition applications, short video social applications, audio and video conferencing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0038] Terminal devices 101, 102, 103 can be hardware or software. When terminal devices 101, 102, 103 are hardware, they can be various electronic devices with sound acquisition devices (such as microphones), video acquisition devices (such as cameras) and display screens, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) players, laptop computers and desktop computers, etc. When terminal devices 101, 102, 103 are software, they can be installed in the terminal devices listed above. It can be implemented as multiple software or software modules (for example, used to provide image category annotation services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0039] In some cases, the image category labeling method provided by the present disclosure may be executed by the terminal devices 101, 102, 103, and accordingly, the image category labeling apparatus may be disposed in the terminal devices 101, 102, 103. In this case, the system architecture 100 may not include the server 105.

[0040] In some cases, the image category annotation method provided by the present disclosure can be jointly performed by the terminal devices 101, 102, 103 and the server 105. For example, the step of "extracting features from the image to be annotated to obtain features of the image to be annotated" can be performed by the terminal devices 101, 102, 103, and the steps of "obtaining an N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the preset image annotation category set" can be performed by the server 105. The present disclosure does not limit this. Accordingly, the image category annotation device can also be respectively set in the terminal devices 101, 102, 103 and the server 105.

[0041] In some cases, the image category labeling method provided in the present disclosure may be executed by the server 105 , and accordingly, the image category labeling device may also be set in the server 105 . In this case, the system architecture 100 may not include the terminal devices 101 , 102 , and 103 .

[0042] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0043] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0044] Continue to refer Figure 2 , which shows a process 200 of an embodiment of an image category labeling method according to the present disclosure, the image category labeling method comprises the following steps:

[0045] Step 201 : extract features of the image to be annotated to obtain features of the image to be annotated.

[0046] Here, the image feature to be annotated may include at least one N-dimensional feature vector to be annotated, where N is a positive integer. Optionally, the image feature to be annotated may be represented in various data forms, such as in the form of a feature map.

[0047] The image to be annotated can be understood as an image that needs to be annotated with image categories. For example, the image to be annotated can be a training image that needs to be annotated with image categories in a face liveness detection model training scenario.

[0048] In this embodiment, the execution subject of the image category labeling method (for example Figure 1 The server shown in the figure can first obtain the image to be annotated that needs to be annotated. Then, various feature extraction methods can be used to extract features of the image to be annotated to obtain corresponding features of the image to be annotated for subsequent calculation.

[0049] In an optional embodiment, the image to be annotated can be input into a pre-trained feature extraction model to obtain the image features to be annotated corresponding to the image to be annotated. Here, the feature extraction model can be a smaller convolutional neural network trained on a small data set, and optimized to a certain extent by using data enhancement; or it can be a representation learned by the pre-trained model to extract the desired features from the new sample, and then input the features into a new classifier, and restart the training generation, thereby improving the acquisition rate of the image features to be annotated through the feature extraction model, and further speeding up the efficiency of subsequent image category annotation.

[0050] It should be noted that the training of the feature extraction model can be set according to the actual application, and the present disclosure does not impose any restrictions on this.

[0051] Step 202: Obtain an N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the preset image annotation category set.

[0052] The preset image annotation category set can be understood as a set of image annotation categories that are pre-prepared and stored by technicians for annotating image categories. For example, in the face liveness detection model training scenario, the preset image annotation category set may include but is not limited to live face images under strong light, non-live face images under strong light, live face images under weak light, non-live face images under weak light, etc.

[0053] The preset image annotation category set can also be understood as a pre-trained image annotation category set for annotating image categories. In addition, the number of image annotation categories in the set can be set as needed, and image annotation categories can be added, deleted or updated based on the original ones according to actual needs.

[0054] Specifically, here, the execution subject may obtain the N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the preset image annotation category set locally or remotely from other electronic devices.

[0055] The N-dimensional image annotation category feature vector corresponding to the preset image annotation category is used to represent a feature vector in the image features obtained by extracting features from images belonging to the image annotation category.

[0056] Here, a preset image annotation category feature vector training method can be used to pre-train to generate an image annotation category feature vector, and the image annotation category feature vector can be pre-stored. Then, in step 202, the N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the preset image annotation category set can be directly obtained to speed up the image category annotation speed.

[0057] In one implementation, the N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the preset image annotation category set can be obtained by Figure 3 The training step 300 shown is obtained in advance, and the training step 300 includes the following steps:

[0058] Step 301: Obtain an N-dimensional initial image annotation category feature vector corresponding to each image annotation category in a sample image set and a preset image annotation category set.

[0059] Here, various parameter initialization methods may be used to initialize the parameters of the N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the preset image annotation category set. For example, a random number between 0 and 1 may be assigned.

[0060] Step 302: Obtain model structures and model parameter information of an initial feature extraction model, an initial image generation model, and an initial image category discrimination model.

[0061] Here, the initial feature extraction model is used to extract image features from the image. For example, the initial feature extraction model may be a convolutional neural network.

[0062] The initial image generation model is used to reconstruct the image features and generate an image. For example, the initial image generation model may be a convolutional neural network.

[0063] The initial image category discrimination model is used to characterize the correspondence between the image and the image category discrimination result, wherein the image category discrimination result is the original image discrimination result or the reconstructed image discrimination result, and the original image discrimination result and the reconstructed image discrimination result are used to indicate that the input image is the original image or the reconstructed image, respectively.

[0064] Step 303 : Based on the sample image set, the N-dimensional image annotation category feature vector, the initial feature extraction model, the initial image generation model and the initial image category discrimination model corresponding to each preset image annotation category in the preset image annotation category set are trained.

[0065] Here, a machine learning method can be used to train the N-dimensional image annotation category feature vector, the initial feature extraction model, the initial image generation model, and the initial image category discrimination model corresponding to each preset image annotation category based on a sample image set.

[0066] The sample image set may be a set consisting of a plurality of sample images, such as a sample image set consisting of a plurality of living face sample images and / or a plurality of non-living face sample images.

[0067] In an optional implementation, step 303 may be performed as follows: for the sample images in the sample image set, a parameter adjustment operation is performed until a preset training end condition is met, where the parameter adjustment operation may include the following steps 3031 to 3035:

[0068] Step 3031: input the sample image into an initial feature extraction model to obtain sample image features of the sample image.

[0069] Here, the initial feature extraction model can be any model for implementing feature extraction, for example, any convolutional neural network. The sample image feature of the sample image includes at least one N-dimensional sample image feature vector.

[0070] Step 3032, for at least one sample image feature vector in the sample image features of the sample image, based on the initial image annotation category feature vector in each initial image annotation category feature vector that is closest to the sample image feature vector, generate an image annotation category coding feature vector for a corresponding position in the image annotation category coding feature corresponding to the sample image, wherein the image annotation category coding feature vector is an N-dimensional vector.

[0071] Specifically, for each sample image vector in the sample image features of the sample image, the vector distance between each initial image annotation category feature vector and the sample image feature vector can be calculated respectively, and then based on the initial image annotation category feature vector that is closest to the sample image feature vector in each initial image annotation category feature vector, an image annotation category encoding feature vector at a corresponding position in the image annotation category encoding feature corresponding to the sample image is generated, wherein each initial image annotation category feature vector corresponds to an image annotation category.

[0072] Step 3031 and step 3032 can be understood as performing an encoding operation on the sample image through the initial feature extraction model and each initial image annotation category feature vector to obtain the image annotation category encoding feature corresponding to the sample image.

[0073] Step 3033: input the image annotation category coding features corresponding to the sample image into the initial image generation model to obtain a reconstructed image corresponding to the sample image.

[0074] Step 3033 can be understood as performing a decoding operation on the image annotation category coding features corresponding to the sample image through the initial image generation model to obtain a reconstructed image corresponding to the sample image.

[0075] Step 3034: input the reconstructed image corresponding to the sample image into the initial image category discrimination model to obtain an image category discrimination result corresponding to the reconstructed image of the sample image.

[0076] After step 3034, an image category discrimination result can be obtained, and the image category discrimination result can be used to determine whether the reconstructed image of the sample image is a reconstructed image or an original image. In practice, the similarity between the reconstructed image and the original image can be calculated through the initial image category discrimination model, and then the image category discrimination result of the reconstructed image of the sample image can be determined.

[0077] Step 3035, based on at least one of the first loss function, the second loss function and the third loss function, adjust the model parameters of the N-dimensional initial image annotation category feature vector, the initial feature extraction model, the initial image generation model and the initial image category discrimination model corresponding to each image annotation category in the preset image annotation category set.

[0078] Among them, the first loss function is used to characterize the difference between the sample image features and the image annotation category coding features of the sample image, the second loss function is used to characterize the difference between the sample image and the corresponding reconstructed image, and the third loss function is used to characterize the difference between the image category discrimination result corresponding to the sample image and the reconstructed image discrimination result.

[0079] Here, the N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the preset image annotation category set can be optimized through the first loss function, the model parameters of the initial feature extraction model can be optimized through the second loss function, and the model parameters of the initial image generation model and the initial image category discrimination model can be optimized through the third loss function.

[0080] It should be noted that, when the sample images in the sample image set are input into the initial feature extraction model in step 3031, one sample image may be input each time, and then subsequent parameter adjustment operations may be performed through the one sample image until the preset training end condition is met; or at least two sample images may be input, and then subsequent parameter adjustment operations may be performed based on the at least two sample images, until the preset training end condition is met. The present disclosure does not make specific limitations on this.

[0081] Here, the preset training end condition may be various preset conditions for determining model convergence. For example, the preset training condition may include at least one of the following conditions:

[0082] Condition 1: The number of times step 3035 is executed is greater than or equal to a preset number of times.

[0083] Condition 2: The time for executing step 3035 exceeds the preset training time.

[0084] Condition 3: the first loss function, the second loss function, and the third loss function obtained in step 3035 are less than their respective corresponding preset loss function thresholds.

[0085] Condition 4, before step 3035, a verification data set is obtained in advance. Among them, the verification data in the verification data set includes the N-dimensional initial image annotation category feature vector corresponding to each verification image annotation category in the verification image set and the verification image annotation category set. Moreover, the verification image set corresponding to the verification data set can be the same as or completely different from the sample image set. Then, the difference between the first loss function, the second loss function and the third loss function obtained in this step 3035 and the first loss function, the second loss function and the third loss function calculated when the model parameters of the N-dimensional initial image annotation category feature vector, the initial feature extraction model, the initial image generation model and the initial image category discrimination model were adjusted last time is calculated. Condition 4 is that the difference obtained by the above calculation is less than the preset loss function difference threshold. That is, the loss function of the N-dimensional initial image annotation category feature vector, the initial feature extraction model, the initial image generation model and the initial image category discrimination model no longer decreases or decreases very little on the verification data set.

[0086] Step 304 : Determine the trained N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the preset image annotation category set as the N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the preset image annotation category set.

[0087] Continue to refer Figure 4 , which shows a schematic diagram of an application scenario of the training step according to the present disclosure.

[0088] exist Figure 4 In the application scenario, first, a sample face image 401 is input into the initial feature extraction model 402 to obtain a sample face image feature 403 of the sample face image 401, and the size of the sample face image feature 403 is H×W×D; H and W represent the length and width of the sample image of the sample face image feature 403, and the sample face image feature 403 includes H×W D-dimensional sample image feature vectors.

[0089] The preset image annotation category set stores K preset image annotation categories, and each preset image annotation category corresponds to a D-dimensional initial face image annotation category feature vector, and the initial face image annotation category feature vector corresponding to each preset image annotation category is represented by e1, e2, e3, ..., ek. The preset face image annotation category set can be stored in the form of a database or a data table, which is not specifically limited in the present disclosure.

[0090] Then, for the sample face image feature vector z contained in the sample face image feature 403 e(x) can be used to find the initial face image annotation category feature vector closest to itself in the preset face image annotation category set, that is, the sample face feature vector z contained in the sample face image feature e (x) is further discretized and encoded. Then the subscript of the category feature vector of the retrieved initial face image is used to indicate the corresponding category, i.e. q(z|x). The discrete encoding process of this step can be expressed as:

[0091]

[0092] Among them, e i is the initial face image annotation category feature vector corresponding to the i-th preset face image annotation category, z q (x) is the sample image feature vector after encoding, H×W D-dimensional sample image feature vector after encoding vector z q (x) can form the image annotation category coding feature 404.

[0093] The image annotation category coding feature 404 is input into the face image generation model 405 to obtain a reconstructed face image 406, and then the reconstructed face image 406 is input into the image type discrimination model 407 to determine whether the current reconstructed face image is the original sample image or comes from the reconstructed face image.

[0094] Next, the above step 3035 can be executed to adjust the model parameters of the N-dimensional initial image annotation category feature vectors e1, e2, e3, ..., ek, the initial feature extraction model 402, the initial image generation model 405 and the initial image category discrimination model 407 corresponding to each image annotation category in the preset image annotation category set.

[0095] It should be noted that, when subsequently training a new face liveness detection model, the face image annotation categories of the sample face images can be used as additional face image annotation categories to supervise the training process of the face liveness detection model.

[0096] Step 203 : determining the image annotation category of the image to be annotated based on the image annotation category corresponding to the feature vector to be annotated.

[0097] The image annotation category corresponding to the feature vector to be annotated is the preset image annotation category whose image annotation category feature vector corresponding to each preset image category is closest to the feature vector to be annotated. Specifically, for the feature vector to be annotated in the image feature to be annotated, the vector distance between the feature vector to be annotated and the image annotation category feature vector corresponding to each preset image annotation category can be calculated, and then the image annotation category in each preset image annotation category that is closest to the feature vector to be annotated is determined as the image annotation category of the feature vector to be annotated.

[0098] In one implementation, the image annotation categories corresponding to multiple feature vectors to be annotated in the image features to be annotated may be integrated to determine the image annotation category corresponding to the image to be annotated. The specific implementation is as follows:

[0099] First, for each feature vector to be annotated, determine the image annotation category corresponding to the feature vector to be annotated;

[0100] Then, the image annotation categories corresponding to each feature vector to be annotated are merged;

[0101] Finally, the image annotation categories obtained after merging are determined as the image annotation categories of the image to be annotated.

[0102] For example, a sample image corresponds to an image feature to be labeled, and the image feature to be labeled includes 4 feature vectors to be labeled. The image annotation categories corresponding to the 4 feature vectors to be labeled are determined to be: "strong light", "unobstructed", "strong light" and "real person", and then the 4 image annotation categories are merged, and the two image annotation categories of "strong light" and "strong light" are merged into one, so as to obtain the image annotation categories corresponding to the sample image as "strong light", "unobstructed" and "real person".

[0103] In another implementation, the number of times the image annotation category appears can be counted, and then the image annotation category corresponding to the image to be annotated can be determined according to the number of times the image annotation category appears. The specific implementation is as follows:

[0104] First, for each feature vector to be annotated, the image annotation category corresponding to the feature vector to be annotated is determined.

[0105] Then, the number of occurrences of each preset image annotation category in the image annotation category corresponding to each feature vector to be annotated is counted.

[0106] Finally, the image annotation category of the image to be annotated is determined according to the number of occurrences of each preset image annotation category obtained by statistics.

[0107] Specifically, the statistically obtained occurrence counts of each preset image annotation category may be sorted, and then the preset image annotation category with the largest occurrence count may be determined as the image annotation category of the image to be annotated.

[0108] Or in one implementation, a preset image annotation category in the preset image annotation category set whose corresponding occurrence times are greater than a preset number threshold is determined as the image annotation category of the image to be annotated.

[0109] For example, the preset number threshold is set to 3 times, and then the preset image annotation category with a corresponding appearance number greater than 3 times in the preset image annotation category set is determined by statistics to be determined as the image annotation category of the image to be annotated.

[0110] For example, for a sample image, its corresponding image feature to be annotated includes 5 feature vectors to be annotated. The annotation categories corresponding to these 5 feature vectors to be annotated are: "strong light", "real person", "strong light", "fake person" and "strong light". It can be determined that the annotation category "strong light" appears 3 times, "real person" appears 1 time, and "fake person" appears 1 time. Based on the preset number threshold of 3 times, the image annotation category "strong light" is determined as the image annotation category corresponding to the sample image.

[0111] In an optional implementation, when there are at least two preset image annotation categories whose occurrence times are greater than a preset number threshold, the multiple preset image annotation categories can be integrated and the integrated preset image annotation category can be determined as the image annotation category of the image to be annotated.

[0112] In another optional implementation, the first preset number of preset image annotation categories with the largest number of corresponding occurrences in the preset image annotation set may be determined as the image annotation categories of the image to be annotated.

[0113] For example, the number of occurrences of each preset image annotation category in the preset image annotation set is sorted, and according to the sorting result, the first three corresponding preset image annotation categories with the largest number of occurrences are determined as the image annotation categories of the image to be annotated, where the preset number can be set according to actual applications, and the present disclosure does not impose any limitation on this.

[0114] The image category labeling method, device, electronic device and storage medium provided by the embodiments of the present disclosure extract features of the image to be labeled to obtain the image features to be labeled, then obtain the image labeling category feature vector corresponding to each preset image labeling type in the preset image labeling category set, and determine the preset image labeling category whose distance between the feature vector to be labeled included in the image features to be labeled and the image labeling category feature vector is closest, and then determine it as the image labeling category of the image to be labeled. The fine-grainedness of image labeling can be improved. Compared with the binary labeling of traditional detection tasks, the automation of high-grained image category labeling is realized, and the efficiency of image labeling category determination is improved. Subsequently, the distribution characteristics of the existing image set can be comprehensively analyzed through the labeled image, and the image can be targetedly expanded according to the needs of the training task. The model is iteratively updated using the expanded image data, so as to achieve targeted improvement of model capabilities and further improve the generalization of the model.

[0115] Continue to refer Figure 5 , which shows a process 500 of another embodiment of the image category labeling method according to the present disclosure. The image category labeling method, applied to the human face live image labeling scenario, includes the following steps:

[0116] Step 501: Obtain a live face image to be annotated that needs image category annotation.

[0117] Step 502: input the live face image to be annotated into a feature extraction model to obtain live face image features corresponding to the live face image to be annotated.

[0118] The features of the live face image to be annotated can be understood as the features of the image to be annotated mentioned above, and can be represented in the form of a feature map.

[0119] Step 503: Obtain a live face image annotation category feature vector corresponding to each preset live face image annotation category in the preset live face image annotation category set.

[0120] Step 504: Calculate the distance between each live face feature vector to be annotated in the live face image features to be annotated and each live face image annotation category feature vector corresponding to each preset live face image annotation category.

[0121] Step 505: According to the calculated distance information, determine the preset live face image annotation category corresponding to the live face image category feature vector closest to the live face feature vector to be annotated, and determine the live face annotation category of the live face image to be annotated.

[0122] Specifically, after determining the face liveness annotation category of the face liveness image feature to be annotated, the face liveness image feature can be used as a sample, and its corresponding face liveness annotation category can be used as the corresponding sample label, which is input into the face liveness detection model to be trained to obtain the face liveness detection model, and then the face liveness detection is performed according to the face liveness detection model.

[0123] The specific implementation of steps 501-505 is basically consistent with the specific implementation and technical effects of the image category labeling method in the above embodiment, which will not be repeated here. For details, please refer to the image category labeling method in the above embodiment.

[0124] The image category labeling method, device, electronic device and storage medium provided by the embodiments of the present disclosure extract features of the image to be labeled to obtain the image features to be labeled, then obtain the image labeling category feature vector corresponding to each preset image labeling type in the preset image labeling category set, and determine the preset image labeling category whose distance between the feature vector to be labeled included in the image features to be labeled and the image labeling category feature vector is closest, and then determine it as the image labeling category of the image to be labeled. The fine-grainedness of image labeling can be improved. Compared with the binary labeling of traditional detection tasks, the automation of high-grained image category labeling is realized, and the efficiency of image labeling category determination is improved. Subsequently, the distribution characteristics of the existing image set can be comprehensively analyzed through the labeled image, and the image can be targetedly expanded according to the needs of the training task. The model is iteratively updated using the expanded image data, so as to achieve targeted improvement of model capabilities and further improve the generalization of the model.

[0125] Further references Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an image category labeling device. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0126] like Figure 6 As shown, the image category labeling device 600 of this embodiment includes: an image acquisition unit 601, a vector acquisition unit 602 and a vector determination unit 603. The image acquisition unit 601 is configured to extract features of the image to be labeled to obtain features of the image to be labeled, wherein the image features to be labeled include at least one N-dimensional feature vector to be labeled, and N is a positive integer; the vector acquisition unit 602 is configured to obtain an N-dimensional image labeling category feature vector corresponding to each preset image labeling category in the preset image labeling category set; the vector determination unit 603 is configured to determine the image labeling category of the image to be labeled based on the image labeling category corresponding to the feature vector to be labeled, wherein the image labeling category corresponding to the feature vector to be labeled is the preset image labeling category whose image labeling category feature vector corresponding to each of the preset image categories is closest to the feature vector to be labeled.

[0127] In this embodiment, the specific processing of the image acquisition unit 601, the vector acquisition unit 602, and the vector determination unit 603 of the image category labeling device 600 and the technical effects thereof can be referred to in Figure 2 The relevant descriptions of step 201, step 202 and step 203 in the corresponding embodiment are not repeated here.

[0128] In an optional embodiment, the N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the above-mentioned preset image annotation category set is pre-obtained through the following training steps: obtaining a sample image set and an N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the above-mentioned preset image annotation category set; obtaining the model structure and model parameter information of the initial feature extraction model, the initial image generation model and the initial image category discrimination model; based on the above-mentioned sample image set, the adjusted N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the above-mentioned preset image annotation category set, the above-mentioned initial feature extraction model, the above-mentioned initial image generation model and the above-mentioned initial image category discrimination model are trained; and the trained N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the above-mentioned preset image annotation category set is determined as the N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the above-mentioned preset image annotation category set.

[0129] In an optional embodiment, the image category labeling device further includes a parameter adjustment unit, which is further configured to: perform parameter adjustment operations on the sample images in the sample image set, the initial feature extraction model, the initial image generation model, and the model parameter information of the initial image category discrimination model, including: for the sample images in the sample image set, perform the following parameter adjustment operations until the preset training end condition is met: input the sample image into the initial feature extraction model to obtain the image features of the sample image, wherein the image features of the sample image include at least one N-dimensional sample image feature vector; for each sample image feature vector in the image features of the sample image, use the initial image labeling category feature vector that is closest to the sample image feature vector in each of the initial image labeling category feature vectors to generate an image labeling category coding feature vector at a corresponding position in the image labeling category coding feature corresponding to the sample image, wherein the image labeling category coding feature vector is an N-dimensional vector; input the image labeling category coding feature corresponding to the sample image into the initial image generation model model, obtain a reconstructed image corresponding to the sample image; input the reconstructed image corresponding to the sample image into the above-mentioned initial image category discrimination model, and obtain an image category discrimination result corresponding to the reconstructed image of the sample image, wherein the image category discrimination result is an original image discrimination result or a reconstructed image discrimination result, and the above-mentioned original image discrimination result and the above-mentioned reconstructed image discrimination result are respectively used to indicate that the input image is an original image or a reconstructed image; based on at least one of the first loss function, the second loss function and the third loss function, adjust the N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the above-mentioned preset image annotation category set, the model parameters of the above-mentioned initial feature extraction model, the above-mentioned initial image generation model and the above-mentioned initial image category discrimination model, wherein the above-mentioned first loss function is used to characterize the difference between the image feature of the above-mentioned sample image and the image annotation category coding feature, the above-mentioned second loss function is used to characterize the difference between the sample image and the corresponding reconstructed image, and the above-mentioned third loss function is used to characterize the difference between the image category discrimination result corresponding to the sample image and the above-mentioned reconstructed image discrimination result.

[0130] In some optional embodiments, the vector determination unit 603 is further configured to: determine the image annotation category of the image to be annotated based on the image annotation category corresponding to the feature vector to be annotated, including: for each feature vector to be annotated in at least one of the feature vectors to be annotated, determine the image annotation category corresponding to the feature vector to be annotated; merge the image annotation category corresponding to each feature vector to be annotated in at least one of the feature vectors to be annotated; and determine the image annotation categories obtained after the merger as the image annotation category of the image to be annotated.

[0131] In some optional embodiments, the vector determination unit 603 is further configured to: determine the image annotation category of the image to be annotated based on the image annotation category corresponding to the feature vector to be annotated, including: for each feature vector to be annotated in each of the feature vectors to be annotated, determine the image annotation category corresponding to the feature vector to be annotated; count the number of occurrences of each of the preset image annotation categories in the image annotation categories corresponding to each of the feature vectors to be annotated; and determine the image annotation category of the image to be annotated according to the number of occurrences of each of the preset image annotation categories obtained by counting.

[0132] In some optional embodiments, the vector determination unit 603 is further configured to determine the image annotation category of the image to be annotated based on the statistically obtained number of occurrences of each of the preset image annotation categories, including: determining the preset image annotation category in the preset image annotation category set whose corresponding number of occurrences is greater than a preset number threshold as the image annotation category of the image to be annotated.

[0133] In some optional embodiments, the vector determination unit 603 is further configured to: determine the image annotation category of the image to be annotated according to the statistically obtained number of occurrences of each of the preset image annotation categories, including: determining the first preset number of preset image annotation categories with the largest number of corresponding occurrences in the preset image annotation category set as the image annotation category of the image to be annotated.

[0134] In some optional implementations, the preset image annotation category set includes image categories in a face liveness detection scenario, and the images to be annotated include images that need to be annotated with image categories in a face liveness detection scenario.

[0135] It should be noted that the implementation details and technical effects of each unit in the image category labeling device provided in the embodiments of the present disclosure can be referred to the description of other embodiments in the present disclosure, and will not be repeated here.

[0136] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 suitable for implementing the electronic device of the present disclosure. Figure 7 The computer system 700 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0137] like Figure 7As shown, the computer system 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 to a random access memory (RAM) 703. Various programs and data required for the operation of the computer system 700 are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0138] Typically, the following devices may be connected to the I / O interface 705: input devices 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, etc.; output devices 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 708 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 709. The communication devices 709 may allow the computer system 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The computer system 700 of the electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0139] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0140] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0141] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0142] The computer readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device implements the following Figure 2 The image category labeling method shown in the embodiment and its optional implementation manner, and / or, as Figure 5 The illustrated embodiment and its optional implementations illustrate a method for labeling image categories.

[0143] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0144] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0145] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not limit the unit itself in some cases.

[0146] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

Claims

1. A method for labeling image categories, comprising: Extracting features from the image to be annotated to obtain features of the image to be annotated, wherein the features of the image to be annotated include at least one N-dimensional feature vector to be annotated, where N is a positive integer; Obtain an N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the preset image annotation category set; Based on the image annotation category corresponding to the feature vector to be annotated, the image annotation category of the image to be annotated is determined, wherein the image annotation category corresponding to the feature vector to be annotated is the preset image annotation category whose corresponding image annotation category feature vector in each of the preset image categories is closest to the feature vector to be annotated.

2. The image category labeling method according to claim 1, wherein: The N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the preset image annotation category set is obtained in advance through the following training steps: Obtaining an N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the sample image set and the preset image annotation category set; Obtaining model structure and model parameter information of an initial feature extraction model, an initial image generation model, and an initial image category discrimination model; Based on the sample image set, training the N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the preset image annotation category set, the initial feature extraction model, the initial image generation model, and the initial image category discrimination model; The trained N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the preset image annotation category set is determined as the N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the preset image annotation category set.

3. The image category labeling method according to claim 2, wherein: The training of the N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the preset image annotation category set, the initial feature extraction model, the initial image generation model, and the initial image category discrimination model based on the sample image set includes: For the sample images in the sample image set, the following parameter adjustment operations are performed until the preset training end condition is met: Inputting the sample image into the initial feature extraction model to obtain a sample image feature of the sample image, wherein the sample image feature of the sample image includes at least one N-dimensional sample image feature vector; For at least one sample image feature vector in the sample image features of the sample image, based on the initial image labeling category feature vector that is closest to the sample image feature vector in each of the initial image labeling category feature vectors, generate an image labeling category coding feature vector at a corresponding position in the image labeling category coding feature corresponding to the sample image, wherein the image labeling category coding feature vector is an N-dimensional vector; Inputting the image annotation category coding feature corresponding to the sample image into the initial image generation model to obtain a reconstructed image corresponding to the sample image; Inputting the reconstructed image corresponding to the sample image into the initial image category discrimination model to obtain an image category discrimination result corresponding to the reconstructed image of the sample image, wherein the image category discrimination result is an original image discrimination result or a reconstructed image discrimination result, and the original image discrimination result and the reconstructed image discrimination result are respectively used to indicate that the input image is an original image or a reconstructed image; Based on at least one of the first loss function, the second loss function and the third loss function, adjusting the N-dimensional initial image annotation category feature vector corresponding to each image annotation category in the preset image annotation category set, the model parameters of the initial feature extraction model, the initial image generation model and the initial image category discrimination model, Among them, the first loss function is used to characterize the difference between the image features of the sample image and the image annotation category coding features, the second loss function is used to characterize the difference between the sample image and the corresponding reconstructed image, and the third loss function is used to characterize the difference between the image category discrimination result corresponding to the sample image and the reconstructed image discrimination result.

4. The image category labeling method according to claim 1, wherein: The determining the image annotation category of the image to be annotated based on the image annotation category corresponding to the feature vector to be annotated includes: For each of the at least one feature vector to be labeled, determining the image labeling category corresponding to the feature vector to be labeled; Merging the image annotation category corresponding to each feature vector to be annotated in at least one of the feature vectors to be annotated; The image annotation categories obtained after merging are determined as the image annotation categories of the image to be annotated.

5. The image category labeling method according to claim 1, wherein: The determining the image annotation category of the image to be annotated based on the image annotation category corresponding to the feature vector to be annotated includes: For each of the at least one feature vector to be labeled, determining the image labeling category corresponding to the feature vector to be labeled; Counting the number of occurrences of each of the preset image annotation categories in the image annotation categories corresponding to each feature vector to be annotated in at least one of the feature vectors to be annotated; The image annotation category of the image to be annotated is determined according to the statistically obtained number of occurrences of each of the preset image annotation categories.

6. The image category labeling method according to claim 5, wherein: Determining the image annotation category of the image to be annotated according to the statistically obtained number of occurrences of each of the preset image annotation categories includes: A preset image annotation category in the preset image annotation category set whose corresponding occurrence frequency is greater than a preset frequency threshold is determined as the image annotation category of the image to be annotated.

7. The image category labeling method according to claim 5, wherein: Determining the image annotation category of the image to be annotated according to the statistically obtained number of occurrences of each of the preset image annotation categories includes: The first preset number of preset image annotation categories with the largest number of corresponding occurrences in the preset image annotation category set are determined as image annotation categories of the image to be annotated.

8. The image category labeling method according to claim 1, wherein: The preset image annotation category set includes image categories in a face liveness detection scenario, and the to-be-annotated images include images that need to be annotated with image categories in a face liveness detection scenario.

9. An image category labeling device, comprising: An image acquisition unit is configured to perform feature extraction on the image to be annotated to obtain features of the image to be annotated, wherein the features of the image to be annotated include at least one N-dimensional feature vector to be annotated, where N is a positive integer; A vector acquisition unit, configured to acquire an N-dimensional image annotation category feature vector corresponding to each preset image annotation category in the preset image annotation category set; The vector determination unit is configured to determine the image annotation category of the image to be annotated based on the image annotation category corresponding to the feature vector to be annotated, wherein the image annotation category corresponding to the feature vector to be annotated is the preset image annotation category whose corresponding image annotation category feature vector in each of the preset image categories is closest to the feature vector to be annotated.

10. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 7.

11. A computer-readable storage medium having a computer program stored thereon, wherein: When the computer program is executed by one or more processors, the method according to any one of claims 1 to 7 is implemented.