Image feature updating method and device, and electronic device

Abstract

The application discloses an image feature updating method and device and electronic equipment. It relates to the field of artificial intelligence, and comprises the following steps: determining a target image recognition model, and obtaining a historical image recognition model corresponding to the target image recognition model; obtaining image features of a target image set in the historical image recognition model from a database to obtain an image feature set; inputting the image feature set into a target generator to obtain a restored image set, wherein the restored image set comprises M restored images; inputting each restored image into the target image recognition model in sequence to obtain updated features of each image feature, and storing M updated features into the target image recognition model. Through the application, the problem that the accuracy of image recognition by the updated recognition model using original image features is low in the case where the original image cannot be obtained in the related art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to an image feature updating method and device and electronic equipment. BACKGROUND

[0002] In the field of modern information technology, recognition models, especially those based on deep learning, have been widely applied in the field of image recognition. These models achieve high-precision recognition functions by extracting features from input images and comparing them with features in the base library. With the continuous progress of image recognition technology and the increasing diversification of application demands, recognition models need to be constantly updated to adapt to new challenges and scenarios.

[0003] In the updating process of recognition models, an important but often overlooked problem is the updating of base library features. Since recognition models usually rely on specific feature extraction methods or algorithms, whenever the model is updated, the image features in the base library also need to be re-extracted accordingly to ensure compatibility with the new model. However, in practical applications, users often cannot provide the original images of the base library every time due to various reasons (such as privacy protection, data loss, etc.), resulting in the inability to obtain relatively accurate image features, and thus reducing the recognition accuracy of the updated model.

[0004] In view of the problem in the related art that the accuracy of image recognition by the updated recognition model using original image features is low when the original images cannot be obtained, no effective solution has been proposed so far. SUMMARY

[0005] The present application provides an image feature updating method, device and electronic equipment to solve the problem in the related art that the accuracy of image recognition by the updated recognition model using original image features is low when the original images cannot be obtained.

[0006] According to an aspect of the present application, a method for updating image features is provided. The method comprises: determining a target image recognition model, and obtaining a historical image recognition model corresponding to the target image recognition model, wherein the target image recognition model and the historical image recognition model are both used to recognize target images in a target image set, the target image recognition model and the historical image recognition model are different in version, the target image set contains M target images, and M is a positive integer; obtaining image features of the target image set in the historical image recognition model from a database to obtain an image feature set, wherein the image feature set contains M groups of image features, and each group of image features corresponds to a target image; inputting the image feature set into a target generator to obtain a restored image set, wherein the restored image set contains M restored images; inputting each restored image into the target image recognition model in sequence to obtain updated features of each image feature, and storing the M updated features into the target image recognition model.

[0007] Optionally, before obtaining the image features of the target image set in the historical image recognition model from the database to obtain the image feature set, the method further comprises: inputting each target image into the historical image recognition model to extract feature information of each target image by the historical image recognition model; obtaining identification information of each target image, combining the feature information and the identification information of each target image as image features of each target image to obtain M groups of image features; and storing the M groups of image features into the database.

[0008] Optionally, before inputting the image feature set into the target generator, the method further comprises: for any target image, judging whether the target image contains text data, wherein the text data is used to describe the target image in a text form; in the case that the target image contains the text data, inputting the text data into a text encoder to obtain first features of the target image; combining the first features of the target image and the image features of the target image to obtain updated image features, and inputting the updated image features into the target generator.

[0009] Optionally, before inputting the image feature set into the target generator, the method further comprises: for any target image, judging whether the target image contains sketch data, wherein the sketch data is used to describe the target image in a sketch form; in the case that the target image contains the sketch data, inputting the sketch data into a sketch encoder to obtain second features of the target image; combining the second features of the target image and the image features of the target image to obtain updated image features, and inputting the updated image features into the target generator.

[0010] Optionally, the text encoder is trained by: obtaining a plurality of first sample images, and sample feature data and sample image description text of each first sample image; combining each first sample image and the corresponding sample feature data and sample image description text into sample data to obtain a plurality of first sample data; training an initial text encoder through the plurality of first sample data until the similarity of initial feature data output by the initial text encoder and the sample feature data is greater than a preset similarity, to obtain the text encoder.

[0011] Optionally, the sketch encoder is trained by: obtaining a plurality of second sample images, and sample feature data and sample sketch data of each second sample image; combining each second sample image and the corresponding sample feature data and sample sketch data into sample data to obtain a plurality of second sample data; training an initial sketch encoder through the plurality of second sample data until the similarity of initial feature data output by the initial sketch encoder and the sample feature data is greater than a preset similarity, to obtain the sketch encoder.

[0012] Optionally, the target generator is trained by: obtaining a plurality of third sample images, and sample feature data of each third sample image; combining each third sample image and the corresponding sample feature data into sample data to obtain a plurality of third sample data; training an initial generator through the plurality of third sample data until the similarity between an image generated by the initial generator according to the sample feature data and the corresponding third sample image is greater than a preset similarity, to obtain the target generator.

[0013] According to another aspect of the present application, an image feature updating device is provided. The device comprises: a determination unit configured to determine a target image recognition model and obtain a historical image recognition model corresponding to the target image recognition model, wherein the target image recognition model and the historical image recognition model are both configured to recognize target images in a target image set, the target image recognition model and the historical image recognition model are different in version, the target image set comprises M target images, and M is a positive integer; a first obtaining unit configured to obtain image features of the target image set in the historical image recognition model from a database to obtain an image feature set, wherein the image feature set comprises M groups of image features, and each group of image features corresponds to a target image; a first input unit configured to input the image feature set into a target generator to obtain a restored image set, wherein the restored image set comprises M restored images; and a second input unit configured to input each restored image into the target image recognition model in sequence to obtain updated features of each image feature, and store M updated features into the target image recognition model.

[0014] According to another aspect of the present application, there is also provided a computer program product comprising a computer program which, when executed by a processor, implements the image feature updating method provided by the preceding embodiments of the present application.

[0015] According to another aspect of the present application, there is also provided an electronic device comprising one or more processors and a memory; the memory stores computer readable instructions, and the processor is configured to execute the computer readable instructions, wherein the computer readable instructions, when executed, implement the image feature updating method provided by the preceding embodiments.

[0016] According to the present application, the following steps are adopted: determining a target image recognition model, and obtaining a historical image recognition model corresponding to the target image recognition model, wherein the target image recognition model and the historical image recognition model are both used to recognize target images in a target image set, the target image recognition model and the historical image recognition model are different in version, the target image set contains M target images, and M is a positive integer; obtaining image features of the target image set in the historical image recognition model from a database to obtain an image feature set, wherein the image feature set contains M groups of image features, and each group of image features corresponds to a target image; inputting the image feature set into a target generator to obtain a restored image set, wherein the restored image set contains M restored images; inputting each restored image into the target image recognition model in sequence to obtain updated features of each image feature, and storing the M updated features into the target image recognition model. The present application solves the problem in the related art that the accuracy of image recognition by the updated recognition model using original image features is low in the case where the original image cannot be obtained. By obtaining the image feature set in the historical image recognition model, and restoring each target image through the target generator according to the image feature set to obtain the restored image of each target image, and performing feature extraction on the restored image according to the target image recognition model to obtain the image feature of the original image corresponding to each image feature and recognized by the target image recognition model, the updating operation of the image feature of the target image is completed, and the M updated features are stored into the target image recognition model, thereby achieving the technical effect of improving the recognition accuracy of the target image recognition model on the target image. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and are used to interpret the illustrative embodiments of the present application and their descriptions, and are not intended to be an improper limitation of the present application. In the drawings:

[0018] Figure 1 is a flowchart of the image feature updating method provided by an embodiment of the present application;

[0019] Figure 2is a flowchart of an optional method for generating a restored image according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of an image feature updating apparatus according to an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other without conflict. The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings and in combination with the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to include only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0025] It should be noted that the image feature updating method, device and electronic device determined by the present disclosure can be used in the field of artificial intelligence, and can also be used in any field other than the field of artificial intelligence. The application field of the image feature updating method, device and electronic device determined by the present disclosure is not limited.

[0026] It should be noted that the information collected, user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) used in the present application are all information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards in the relevant region, necessary security measures are taken, it does not violate public order and good customs, and provides corresponding operation portal for users to choose to authorize use or refuse to use, if the user chooses to refuse, enter the expert decision-making process. For example, the system and the related users or institutions are provided with an interface, before obtaining the relevant information, the interface needs to send a request to the aforementioned user or institution, and after receiving the consent information feedback from the aforementioned user or institution, the relevant information is obtained.

[0027] The embodiments or examples of the present disclosure are not exhaustive, but only a part of the embodiments or examples, and are not specific limitations on the protection scope of the present disclosure. In the case of no contradiction, each step in a certain embodiment or example can be implemented as an independent example, and the steps can be combined arbitrarily, for example, the scheme after removing part of the steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be exchanged arbitrarily, in addition, the optional mode or optional example in a certain embodiment or example can be combined arbitrarily; in addition, the embodiments or examples can be combined arbitrarily, for example, part or all steps of different embodiments or examples can be combined arbitrarily, a certain embodiment or example can be combined with the optional mode or optional example of other embodiments or examples.

[0028] For ease of description, the following describes some nouns or terms related to the embodiments of the present application:

[0029] Adversarial network generator (Generative Adversarial Network, GAN): a kind of deep learning model, composed of generator and discriminator. The generator is responsible for generating data, and the discriminator is responsible for judging the difference between the generated data and the real data. Both of them are mutually antagonistic and learn from each other, so that the generator continuously optimizes the quality of the generated data, and finally achieves the effect of generating data close to the real data.

[0030] Encoder: An encoder is used to convert input data into a more abstract, compact representation in order to better capture important features in the input data. An encoder is typically composed of multiple layers, each of which maps the input data to a hidden representation through learned weights. This hidden representation, often referred to as an encoding or feature vector, can preserve key information of the input data and can be used to solve various machine learning tasks such as classification, clustering, generation, etc.

[0031] According to an embodiment of the present application, an image feature updating method is provided.

[0032] Figure 1 is a flowchart of the image feature updating method provided according to an embodiment of the present application. As shown in Figure 1 the method comprises the following steps:

[0033] Step S101, determine a target image recognition model, and obtain a historical image recognition model corresponding to the target image recognition model, wherein the target image recognition model and the historical image recognition model are both used to recognize target images in a target image set, the target image recognition model and the historical image recognition model are different in version, the target image set contains M target images, and M is a positive integer.

[0034] It should be noted that the target image set can be a set of multiple target images, each target image can be a portrait or an object image, and the historical image recognition model can be an initial image recognition model obtained by training using original images of target images. The historical image recognition model can compare the features of an image to be recognized with the features of each target image, respectively, to determine whether the image corresponds to a certain target image, and then complete the recognition of the input image by outputting the label of the target image.

[0035] For example, when the historical image recognition model receives a certain portrait, it determines whether the features of the portrait are the same as the features of an image of an employee in an organization, and then determines that the received portrait is the portrait of the employee when they are the same, and then outputs the name of the employee, thereby completing the character recognition operation.

[0036] It should be noted that the target image recognition model can be an updated historical image recognition model, and the target image recognition model can improve the accuracy of image recognition by modifying the model structure, model parameters, etc.

[0037] Specifically, after the target image recognition model is trained, in order to enable the target image recognition model to recognize the target images in the target image set, the image features of each target image in the target image set need to be input into the target image recognition model. At this time, the historical image recognition model corresponding to the target image recognition model can be acquired first, and then the image features of the target image are acquired from the historical image recognition model, and the image features applied to the target image recognition model are obtained by processing the image features in the historical image recognition model.

[0038] In step S102, the image features of the target image set in the historical image recognition model are acquired from the database to obtain an image feature set, wherein the image feature set includes M groups of image features, and each group of image features corresponds to a target image.

[0039] Specifically, after the historical image recognition model is acquired, the image features of the target image set in the historical image recognition model can be acquired from the database of the server running the historical image recognition model, wherein the image features can also be stored in the model, and the storage location is not limited here.

[0040] After the image features are obtained, the multiple image features can be divided according to the target images corresponding to the image features to obtain an image feature set, and each group of image features in the set corresponds to a target image, so that the corresponding target image can be restored according to each image feature to obtain a restored image, and then the image feature recognition operation can be performed again according to the restored image, and the image feature updating operation can be completed.

[0041] In step S103, the image feature set is input into the target generator to obtain a restored image set, wherein the restored image set includes M restored images.

[0042] Specifically, after the image feature set is obtained, a group of image features corresponding to each target image can be input into the target generator in sequence, so that the image restoration operation is performed by the target generator according to the image features to obtain the restored image corresponding to each target image.

[0043] In step S104, each restored image is input into the target image recognition model in sequence to obtain updated features of each image feature, and the M updated features are stored in the target image recognition model.

[0044] Specifically, after obtaining each restored image, since the restored image contains most image features of the original image of the target image, each restored image can be input into the target image recognition model, so that the image features in each restored image are recognized by the encoder in the target image recognition model, so that more accurate image features are recognized from the restored image by the updated target image recognition model to obtain updated features, and then the updated features are stored in the target image recognition model or in the database, so that when the target image recognition model is used to identify the real person or object corresponding to the target image, more accurate identification can be performed according to the updated features, thereby achieving the updating of the image features and ensuring the accuracy of the object identification by the target image recognition model.

[0045] The image feature updating method provided by the embodiment of the application comprises the following steps: determining a target image recognition model, and obtaining a historical image recognition model corresponding to the target image recognition model, wherein the target image recognition model and the historical image recognition model are both used to identify target images in a target image set, the target image recognition model and the historical image recognition model are different in version, the target image set comprises M target images, and M is a positive integer; obtaining image features of the target image set in the historical image recognition model from a database to obtain an image feature set, wherein the image feature set comprises M groups of image features, and each group of image features corresponds to a target image; inputting the image feature set into a target generator to obtain a restored image set, wherein the restored image set comprises M restored images; inputting each restored image into the target image recognition model in sequence to obtain updated features of each image feature, and storing the M updated features in the target image recognition model. The method solves the problem in the related art that the accuracy of image recognition by the updated recognition model using original image features is low when the original image cannot be obtained. By obtaining the image feature set in the historical image recognition model, restoring each target image by the target generator according to the image feature set to obtain a restored image of each target image, and performing feature extraction on the restored image according to the target image recognition model to obtain the image features of the original image recognized by the target image recognition model corresponding to each image feature, the updating of the image features of the target image is completed, and the M updated features are stored in the target image recognition model, thereby achieving the technical effect of improving the recognition accuracy of the target image recognition model for the target image.

[0046] Optionally, in the image feature updating method provided by the embodiments of the present application, before the image features of the target image set in the historical image recognition model are obtained from the database, the method further comprises: inputting each target image into the historical image recognition model to extract the feature information of each target image by the historical image recognition model; obtaining the identification information of each target image, combining the feature information and the identification information of each target image as the image feature of each target image, and obtaining M groups of image features; and storing the M groups of image features into the database.

[0047] It should be noted that the historical image recognition model refers to a deep learning model that has been previously trained and used to recognize and classify face images, for example, a model based on Arcface, Cosface, etc. algorithm, which can extract key face feature information from images and determine whether the current face image is consistent with the image stored in the library according to the face feature information, and then determine whether the user performing face verification passes the verification. The identification information can be the individual identity information corresponding to each target image, which can exist in the form of a label or an ID, and is used to distinguish different face images.

[0048] Specifically, before obtaining the image features, accurate image features need to be generated first. First, each target image in the target image set needs to be input into the historical image recognition model. Through the forward propagation of the model, the feature information of each target image is extracted, so that through the feature extraction capability of the historical model, the features of each image are accurately encoded, providing a basis for subsequent image feature combination and storage.

[0049] After obtaining the image features, in order to determine the target image corresponding to the image features, the identification information of each target image in the target image set needs to be obtained. The identification information can be obtained based on the metadata of the image or the pre-recorded identity label, so as to ensure the clear ownership of each image feature and provide a basis for subsequent image feature management and query.

[0050] After obtaining the identification information, the feature information and the identification information of each target image can be combined to form the image feature of each target image, so that the feature information and the identification information are integrated into a more comprehensive data unit, which contains not only visual features but also identity information. Then, in the subsequent process, the model can operate based on the feature information containing the identity information, ensuring that the generated image not only has visual feature consistency, but also accurately corresponds to a specific individual, and ensuring that the correspondence between each image feature and the original target image is clear when stored, facilitating subsequent retrieval and use.

[0051] The embodiment improves the efficiency and accuracy of subsequent image reconstruction by accurately extracting and combining the feature information and identification information of the target image before obtaining the image features.

[0052] Optionally, in the image feature updating method provided in the embodiment of the application, before the image feature set is input into the target generator, the method further includes: for any one target image, judging whether the target image has text data, wherein the text data is used to describe the target image in a text form; in the case that the target image has the text data, inputting the text data into a text encoder to obtain first features of the target image; combining the first features of the target image and the image features of the target image to obtain updated image features, and inputting the updated image features into the target generator.

[0053] Specifically, in order to improve the accuracy of the target generator in generating the restored image, before the image features are input into the target generator, it can be determined whether the target image corresponding to the image features has text data, wherein the text data can be a paragraph of text used to describe the target image, and whether the target image has additional text description information can be determined by querying a database.

[0054] In the case that the text data exists, the text data can be input into a pre-trained text encoder. The text encoder can convert the natural language description into a high-dimensional feature vector, i.e., first features of the target image, so as to obtain the image features corresponding to the target image according to the text data.

[0055] After obtaining the first features of the target image, the first features of the target image can be fused with the image features extracted by the historical image recognition model, for example, weighted average or specific feature splicing in the feature space, so as to combine the image features corresponding to the text data with the pre-stored image features in the model, obtain more accurate updated image features, and further ensure that the generated image contains not only the visual features of the original image but also reflects the attribute information of the text description, and then generate the restored image through the updated image features, thereby achieving the technical effect of improving the accuracy of the restored image.

[0056] The embodiment increases the feature information of the image by adding the checking and fusing steps of the text data before the image feature set is input into the target generator, thereby effectively utilizing the text description information, ensuring the high consistency between the generated image and the base features, and realizing the accurate control and adjustment of the specific attributes of the image through the guidance of the text description, so as to generate more accurate and high-quality restored images.

[0057] Optionally, in the image feature updating method provided in the embodiments of the present application, before the image feature set is input into the target generator, the method further comprises: for any one target image, judging whether the target image has sketch data, wherein the sketch data is used to describe the target image in the form of a sketch; in the case that the target image has the sketch data, inputting the sketch data into a sketch encoder to obtain second features of the target image; combining the second features of the target image and the image features of the target image to obtain updated image features, and inputting the updated image features into the target generator.

[0058] Specifically, in order to improve the accuracy of the target generator in generating the restored image, before the image features are input into the target generator, it can be determined whether the target image corresponding to the image features has sketch data, wherein the sketch data can be an image with low accuracy for describing the target image, and whether the target image has additional sketch data can be determined by querying a database.

[0059] In the case that there is sketch data, the sketch data can be input into a pre-trained sketch encoder. The sketch encoder can extract image features in the sketch to obtain a high-dimensional feature vector, i.e., second features of the target image, so as to obtain the image features corresponding to the target image according to the sketch data.

[0060] After obtaining the second features of the target image, the second features of the target image can be fused with the image features extracted by the historical image recognition model, such as weighted average or specific feature splicing in the feature space, so as to combine the image features corresponding to the sketch data with the pre-stored image features in the model, obtain more accurate updated image features, and then ensure that the generated image contains not only the visual features of the original image but also the attribute information carried in the sketch, and then generate the restored image through the updated image features, thereby achieving the technical effect of improving the accuracy of the restored image.

[0061] The embodiments add the checking and fusing steps of the sketch data before the image feature set is input into the target generator, so as to effectively utilize the feature information carried in the sketch to supplement the initial feature information of the target image, not only ensure the high consistency between the generated image and the base feature, but also enrich the content of the feature information through the feature information in the sketch, thereby generating a more accurate and high-quality restored image.

[0062] It should be noted that in some scenarios, the first features can be obtained through the text encoder and the second features can be obtained through the sketch encoder, and the first features, the second features and the image features are combined to obtain more accurate and comprehensive feature information of the target image.

[0063] Optionally, in the image feature updating method provided in the embodiments of the present application, the text encoder is trained in the following manner: a plurality of first sample images are obtained, and sample feature data and sample image description text of each first sample image are obtained; each first sample image, the corresponding sample feature data and sample image description text are combined into sample data, and a plurality of first sample data are obtained; the initial text encoder is trained through the plurality of first sample data until the similarity of the initial feature data output by the initial text encoder and the sample feature data is greater than a preset similarity, and the text encoder is obtained.

[0064] It should be noted that the first sample image can be an image sample used to train the text encoder, for example, a face image, the sample feature data can be a high-dimensional image feature vector extracted from the first sample image by a historical image recognition model, the sample image description text can be text information associated with the first sample image, used to describe the features of the face in the image, the initial text encoder is an untrained text encoder model, which is used to convert the text description into a vector representation similar to the image feature data, the initial feature data is a feature vector obtained by encoding the sample image description text by the initial text encoder, and the preset similarity is a threshold used to evaluate the similarity between the initial feature data and the sample feature data. The threshold can be used to determine whether the text encoder training meets the requirements.

[0065] Specifically, before using the text encoder, the text encoder needs to be trained using the first sample data to ensure the accuracy of the text encoder. First, a plurality of first sample images can be obtained, the first sample images can be face images, and sample feature data can be extracted from the first sample images by a historical image recognition model, and sample image description text of the first sample images can be obtained.

[0066] Further, the sample feature data and the sample image description text can be input into the model, the sample image description text can be subjected to feature extraction operation by the text encoder to obtain initial feature data, and the initial feature data can be compared with the sample feature data to obtain a comparison result. When the similarity of the comparison result is greater than the preset similarity, it is determined that the text encoder can accurately generate feature information according to the text information, thereby completing the training operation of the text encoder, obtaining a usable text encoder, and further ensuring the accuracy of the feature information of the text information generated by the text encoder.

[0067] It should be noted that the objective function for training the text encoder can be:

[0068]

[0069] wherein L t is the objective function, xt For text information, x is the target image, E v (·) is a picture encoder, E t (·) is a text encoder, f(·) is a nonlinear mapping network used to align the projection space of the text encoder and the picture encoder.

[0070] The embodiment ensures that the model can accurately extract information corresponding to the facial image features from the text description, not only enhances the flexibility of facial feature reconstruction, but also realizes accurate control of facial attributes through text description, and improves the accuracy and diversity of image reconstruction. The efficient training and application of the text encoder provide technical support for the update of the base library of the face recognition system, especially in the case where the original image cannot be directly obtained, the high-quality facial image can be reconstructed based on the feature data and the text description, thereby improving the accuracy of generating updated image features according to the recovered image.

[0071] Optionally, in the image feature updating method provided by the embodiment of the present application, the sketch encoder is trained in the following manner: a plurality of second sample images, and sample feature data and sample sketch data of each second sample image are obtained; each second sample image and the corresponding sample feature data and sample sketch data are combined into sample data to obtain a plurality of second sample data; the initial sketch encoder is trained through the plurality of second sample data until the similarity between the initial feature data output by the initial sketch encoder and the sample feature data is greater than a preset similarity, and the sketch encoder is obtained.

[0072] Specifically, before using the sketch encoder, the sketch encoder needs to be trained using the second sample data to ensure the accuracy of the sketch encoder. First, a plurality of second sample images can be obtained, the second sample images can be facial images, and sample feature data can be extracted from the second sample images by a historical image recognition model, and sample sketch data of the second sample images can be obtained, wherein the sample sketch data can be a sketch of the target image.

[0073] Further, the sample feature data and the sample sketch data can be input into the model, the sample sketch data is subjected to feature extraction operation by the sketch encoder to obtain initial feature data, and the initial feature data is compared with the sample feature data to obtain a comparison result, and in the case that the similarity of the comparison result is greater than a preset similarity, it is determined that the sketch encoder can accurately generate feature information according to the sketch, thereby completing the training operation of the sketch encoder to obtain a usable sketch encoder, and further ensuring the accuracy of the feature information generated by the sketch encoder.

[0074] It should be noted that the objective function for training the sketch encoder can be:

[0075]

[0076] wherein, L s is the objective function, x s is the sketch information, x is the target image, E v (·) is the picture encoder, E s (·) is the sketch encoder.

[0077] The embodiment ensures that the model can accurately extract information corresponding to the facial image features from the sketch by describing the training process of the sketch encoder in detail, not only enhances the flexibility of facial feature reconstruction, but also realizes accurate control of facial attributes through the sketch, and improves the accuracy and diversity of image reconstruction. The efficient training and application of the sketch encoder provide technical support for the update of the base library of the facial recognition system, especially in the case where the original image cannot be directly obtained, the high-quality facial image can be reconstructed based on the feature data and the sketch, thereby improving the accuracy of generating updated image features according to the recovered image.

[0078] Optionally, in the image feature updating method provided by the embodiment of the application, the target generator is trained by: obtaining a plurality of third sample images and sample feature data of each third sample image; combining each third sample image and the corresponding sample feature data into sample data to obtain a plurality of third sample data; and training the initial generator through the plurality of third sample data until the similarity between the image generated by the initial generator according to the sample feature data and the corresponding third sample image is greater than a preset similarity, to obtain the target generator.

[0079] It should be noted that the third sample image can be a set of images used to train the target generator, including facial images of different styles, to ensure that the generator can learn a wide range of facial image generation patterns. The sample feature data can be a high-dimensional feature vector corresponding to the third sample image extracted by a historical image recognition model, which contains the feature information of the face in the image. The initial generator can be a facial image generation network that has not been optimized before training, and its architecture is based on StyleGAN or other similar generation models, which is used to convert high-dimensional features into facial images.

[0080] Specifically, before using the target generator, the target generator needs to be trained using third sample data to ensure the accuracy of the target generator. First, a plurality of third sample images can be obtained, and the second third sample image can be a facial image. The sample feature data is extracted from the third sample image by a historical image recognition model.

[0081] Further, the third sample image and the sample feature data can be input into the model, a corresponding third sample image is generated according to the sample feature data by the target generator, and the generated image is compared with the corresponding third sample image, so that a comparison result is obtained. If the similarity of the comparison result is greater than a preset similarity, it is determined that the target generator can accurately generate an image according to the feature information, so as to complete the training operation of the target generator, obtain a usable target generator, and further ensure the accuracy of the restored image generated by the target generator.

[0082] It should be noted that the sample feature can also include text features and sketch features. The target generator can be trained by different feature combinations, so as to ensure that the target generator can generate accurate restored images according to different features.

[0083] The embodiment trains the target generator to generate a restored image similar to the original face image according to the input high-dimensional feature data. This not only improves the accuracy of image generation, but also enables the reconstruction of a high-quality face image from feature data without the original image, so as to obtain more accurate image features through the target image recognition model, and ensure the accuracy of face recognition through the target image recognition model.

[0084] Figure 2 is a flowchart of an optional method for generating a restored image provided by an embodiment of the present application, as shown in Figure 2 The image features of the target image can be obtained from the historical image recognition model, the text description of the target image is input into the text encoder to obtain the text features, the sketch of the target image is input into the sketch encoder to obtain the sketch features, and then the image features, the sketch features and the text features are input into the target generator to generate a restored image through the target generator, thereby ensuring the accuracy of the restored image, and indirectly improving the accuracy of the image features obtained by the target image recognition model, thereby improving the recognition accuracy of the target image recognition model for the target image.

[0085] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0086] The embodiment of the present application also provides an image feature updating device. It should be noted that the image feature updating device of the embodiment of the present application can be used to execute the image feature updating method provided by the embodiment of the present application. The image feature updating device provided by the embodiment of the present application is introduced as follows.

[0087] Figure 3 is a schematic diagram of an image feature updating apparatus provided by an embodiment of the present application. As shown in the figure, the apparatus comprises a determination unit 31, a first acquisition unit 32, a first input unit 33, and a second input unit 34. Figure 3

[0088] The determination unit 31 is configured to determine a target image recognition model and acquire a historical image recognition model corresponding to the target image recognition model, wherein the target image recognition model and the historical image recognition model are both used for recognizing target images in a target image set, the target image recognition model and the historical image recognition model are different in version, and the target image set comprises M target images, M being a positive integer.

[0089] The first acquisition unit 32 is configured to acquire image features of the target image set in the historical image recognition model from a database to obtain an image feature set, wherein the image feature set comprises M groups of image features, and each group of image features corresponds to one target image.

[0090] The first input unit 33 is configured to input the image feature set into a target generator to obtain a restored image set, wherein the restored image set comprises M restored images.

[0091] The second input unit 34 is configured to sequentially input each restored image into the target image recognition model to obtain updated features of each image feature, and store M updated features into the target image recognition model.

[0092] ​The image feature updating device provided in the embodiments of the present application determines a target image recognition model by the determining unit 31, and obtains a historical image recognition model corresponding to the target image recognition model, wherein the target image recognition model and the historical image recognition model are both used for recognizing target images in a target image set, the target image recognition model and the historical image recognition model are different in version, the target image set contains M target images, and M is a positive integer; the first obtaining unit 32 obtains image features of the target image set in the historical image recognition model from a database, to obtain an image feature set, wherein the image feature set contains M groups of image features, and each group of image features corresponds to a target image; the first input unit 33 inputs the image feature set into a target generator, to obtain a restored image set, wherein the restored image set contains M restored images; the second input unit 34 inputs each restored image into the target image recognition model in sequence, to obtain updated features of each image feature, and stores the M updated features into the target image recognition model. The problem that the accuracy of image recognition by the updated recognition model using original image features is low in the related art when the original image cannot be obtained is solved. The image feature set in the historical image recognition model is obtained, each target image is restored by the target generator according to the image feature set, the restored image of each target image is obtained, the feature extraction operation is performed on the restored image according to the target image recognition model, to obtain the image feature of the original image corresponding to each image feature and recognized by the target image recognition model, the image feature updating operation on the target image is completed, and the M updated features are stored into the target image recognition model, thereby achieving the technical effect of improving the recognition accuracy of the target image recognition model on the target image.

[0093] Optionally, in the image feature updating device provided in the embodiments of the present application, before the image features of the target image set in the historical image recognition model are obtained from the database to obtain the image feature set, the device further includes: a third input unit configured to input each target image into the historical image recognition model to extract feature information of each target image by the historical image recognition model; a second obtaining unit configured to obtain identification information of each target image, combine the feature information and the identification information of each target image as the image feature of each target image, and obtain M groups of image features; and a storage unit configured to store the M groups of image features into the database.

[0094] Optionally, in the image feature updating apparatus provided by the embodiment of the present application, before the image feature set is input into the target generator, the apparatus further comprises: a first judging unit configured to judge, for any one target image, whether the target image has text data, wherein the text data is used to describe the target image in a text form; a fourth input unit configured to input the text data into the text encoder to obtain first features of the target image, in the case that the target image has the text data; and a first combining unit configured to combine the first features of the target image and the image features of the target image to obtain updated image features, and input the updated image features into the target generator.

[0095] Optionally, in the image feature updating apparatus provided by the embodiment of the present application, before the image feature set is input into the target generator, the apparatus further comprises: a second judging unit configured to judge, for any one target image, whether the target image has sketch data, wherein the sketch data is used to describe the target image in a sketch form; a fifth input unit configured to input the sketch data into the sketch encoder to obtain second features of the target image, in the case that the target image has the sketch data; and a second combining unit configured to combine the second features of the target image and the image features of the target image to obtain updated image features, and input the updated image features into the target generator.

[0096] Optionally, in the image feature updating apparatus provided by the embodiment of the present application, the text encoder is obtained by training in the following manner: a third obtaining unit configured to obtain a plurality of first sample images, and sample feature data and sample image description texts of each first sample image; a third combining unit configured to combine each first sample image, the corresponding sample feature data and the sample image description text into sample data to obtain a plurality of first sample data; and a first training unit configured to train an initial text encoder by using the plurality of first sample data until the similarity between initial feature data output by the initial text encoder and the sample feature data is greater than a preset similarity, thereby obtaining the text encoder.

[0097] Optionally, in the image feature updating apparatus provided by the embodiment of the present application, the sketch encoder is obtained by training in the following manner: a fourth obtaining unit configured to obtain a plurality of second sample images, and sample feature data and sample sketch data of each second sample image; a fourth combining unit configured to combine each second sample image, the corresponding sample feature data and the sample sketch data into sample data to obtain a plurality of second sample data; and a second training unit configured to train an initial sketch encoder by using the plurality of second sample data until the similarity between initial feature data output by the initial sketch encoder and the sample feature data is greater than a preset similarity, thereby obtaining the sketch encoder.

[0098] Optionally, in the image feature updating apparatus provided by the embodiment of the present application, the target generator is trained by the following way: a fifth acquisition unit is configured to acquire a plurality of third sample images and sample feature data of each third sample image; a fifth combination unit is configured to combine each third sample image and the corresponding sample feature data as sample data to obtain a plurality of third sample data; and a third training unit is configured to train the initial generator by using the plurality of third sample data until the similarity between the image generated by the initial generator according to the sample feature data and the corresponding third sample image is greater than a preset similarity, and the target generator is obtained.

[0099] The image feature updating apparatus includes a processor and a memory, and the determination unit 31, the first acquisition unit 32, the first input unit 33, the second input unit 34, etc. are stored in the memory as program units, and the corresponding functions are realized by the processor executing the program units stored in the memory.

[0100] The processor includes a kernel, and the kernel retrieves the corresponding program units from the memory. The kernel can be one or more, and the problem that the accuracy of image recognition by the updated recognition model using the original image features is low in the related art when the original image cannot be obtained is solved by adjusting the kernel parameters.

[0101] The memory can include a non-permanent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory includes at least one memory chip.

[0102] The embodiment of the present application provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize the image feature updating method.

[0103] The embodiment of the present application provides a processor, which is used for running a program, and the program is executed to realize the image feature updating method.

[0104] Figure 4 is a schematic diagram of an electronic device according to the embodiment of the present application, as Figure 4As shown, the embodiment of the present application provides an electronic device, the electronic device 40 comprises a processor, a memory and a program stored on the memory and executable on the processor, when the processor executes the program, the following steps are implemented: determining a target image recognition model, and obtaining a historical image recognition model corresponding to the target image recognition model, wherein the target image recognition model and the historical image recognition model are both used for identifying target images in a target image set, the target image recognition model and the historical image recognition model are different in version, the target image set contains M target images, and M is a positive integer; obtaining image features of the target image set in the historical image recognition model from a database to obtain an image feature set, wherein the image feature set contains M groups of image features, and each group of image features corresponds to a target image; inputting the image feature set into a target generator to obtain a restored image set, wherein the restored image set contains M restored images; inputting each restored image into the target image recognition model in sequence to obtain updated features of each image feature, and storing the M updated features into the target image recognition model. The device in the present application can be a server, a PC, a PAD, a mobile phone, etc.

[0105] The present application also provides a computer program product, when executed on a data processing device, is adapted to execute a program which initializes the following method steps: determining a target image recognition model, and obtaining a historical image recognition model corresponding to the target image recognition model, wherein the target image recognition model and the historical image recognition model are both used for identifying target images in a target image set, the target image recognition model and the historical image recognition model are different in version, the target image set contains M target images, and M is a positive integer; obtaining image features of the target image set in the historical image recognition model from a database to obtain an image feature set, wherein the image feature set contains M groups of image features, and each group of image features corresponds to a target image; inputting the image feature set into a target generator to obtain a restored image set, wherein the restored image set contains M restored images; inputting each restored image into the target image recognition model in sequence to obtain updated features of each image feature, and storing the M updated features into the target image recognition model.

[0106] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0107] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0108] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0109] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or combination thereof.

[0110] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0111] The memory can include non-persistent memory and / or volatile memory, such as a random access memory (RAM) including a cache area for the temporary storage of data. The memory can also include non-volatile memory, such as read only memory (ROM), electrically programmable read only memory (EPROM), or electrically erasable programmable read only memory (EEPROM), for the storage of software that is read during runtime. The memory is an example of computer readable media.

[0112] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0113] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or apparatus that includes a list of elements does not only include those elements, but also includes other elements not explicitly listed, or further includes elements inherent in such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0114] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A method of updating an image feature, characterized by, The method comprises the following steps: determine a target image recognition model, and obtain a historical image recognition model corresponding to the target image recognition model, wherein the target image recognition model and the historical image recognition model are both used to identify target images in a target image set, the target image recognition model and the historical image recognition model are different in version, the target image set contains M target images, and M is a positive integer; obtain image features of the target image set in the historical image recognition model from a database to obtain an image feature set, wherein the image feature set contains M groups of image features, and each group of image features corresponds to a target image; input the image feature set into a target generator to obtain a restored image set, wherein the restored image set contains M restored images; input each restored image into the target image recognition model in sequence to obtain updated features of each image feature, and store M updated features into the target image recognition model; before obtaining the image feature set of the target image set in the historical image recognition model from the database, the method further comprises the following steps:

2. The method of claim 1, wherein, input each target image into the historical image recognition model to extract feature information of each target image through the historical image recognition model; obtain identification information of each target image, combine the feature information and the identification information of each target image into image features of each target image to obtain M groups of image features; and store the M groups of image features into the database. before inputting the image feature set into the target generator, the method further comprises the following steps: for any target image, determine whether the target image contains text data, wherein the text data is used to describe the target image in a text form; in the case that the target image contains the text data, input the text data into a text encoder to obtain first features of the target image; 3. The method of claim 1, wherein, combine the first features of the target image and image features of the target image to obtain updated image features, and input the updated image features into the target generator. before inputting the image feature set into the target generator, the method further comprises the following steps: for any target image, determine whether the target image contains sketch data, wherein the sketch data is used to describe the target image in a sketch form; in the case that the target image contains the sketch data, input the sketch data into a sketch encoder to obtain second features of the target image; 4. The method of claim 2, wherein, combine the second features of the target image and image features of the target image to obtain updated image features, and input the updated image features into the target generator. The text encoder is trained in the following way: obtain a plurality of first sample images, sample feature data and sample image description texts of each first sample image; combine each first sample image, the corresponding sample feature data and the sample image description text into sample data to obtain a plurality of first sample data; The initial text encoder is trained by the plurality of first sample data until similarity between initial feature data output by the initial text encoder and the sample feature data is greater than a preset similarity, and the text encoder is obtained.

5. The method of claim 3, wherein, The sketch encoder is obtained by training in the following manner: A plurality of second sample images, sample feature data and sample sketch data of each second sample image are obtained. Each second sample image, corresponding sample feature data and sample sketch data are combined as sample data, and a plurality of second sample data are obtained. The initial sketch encoder is trained by the plurality of second sample data until similarity between initial feature data output by the initial sketch encoder and the sample feature data is greater than a preset similarity, and the sketch encoder is obtained.

6. The method of claim 1, wherein, The target generator is obtained by training in the following manner: A plurality of third sample images and sample feature data of each third sample image are obtained. Each third sample image and corresponding sample feature data are combined as sample data, and a plurality of third sample data are obtained. The initial generator is trained by the plurality of third sample data until similarity between an image generated by the initial generator according to the sample feature data and the corresponding third sample image is greater than a preset similarity, and the target generator is obtained.

7. An image feature updating apparatus characterized by comprising: Comprise: The determination unit is used for determining a target image recognition model and obtaining a historical image recognition model corresponding to the target image recognition model, wherein the target image recognition model and the historical image recognition model are both used for identifying target images in a target image set, the target image recognition model and the historical image recognition model are different in version, the target image set contains M target images, and M is a positive integer; The first acquisition unit is used for acquiring image features of the target image set in the historical image recognition model from a database to obtain an image feature set, wherein the image feature set contains M groups of image features, and each group of image features corresponds to a target image; The first input unit is used for inputting the image feature set into a target generator to obtain a restored image set, wherein the restored image set contains M restored images; The second input unit is used for inputting each restored image into the target image recognition model in sequence to obtain updated features of each image feature, and storing M updated features into the target image recognition model. The device further comprises: a third input unit configured to input each target image into a historical image recognition model to extract feature information of each target image by the historical image recognition model; a second acquisition unit configured to acquire identification information of each target image, combine the feature information and the identification information of each target image as image features of each target image, and obtain M groups of image features; and a storage unit configured to store the M groups of image features into the database.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium includes a stored executable program, wherein the executable program, when executed, controls a device in which the computer readable storage medium is located to execute the image feature update method of any one of claims 1 to 6.

9. An electronic device, comprising: comprise: a memory storing an executable program; a processor configured to execute the program, wherein the program, when executed, performs the image feature update method of any one of claims 1 to 6.

Images