Feature extraction network training and image recognition methods and devices

Abstract

The embodiment of the present application provides a method and device for feature extraction network training and image recognition. The scheme is as follows: using a preset feature extraction network, the first sample image is subjected to feature extraction to obtain first feature data; the first feature data of the first sample image is input into a preset feature conversion network to generate second feature data; according to the recognition results of the preset recognition network for each first sample image corresponding to the first feature data and the second feature data, as well as the first true label, the parameters of the preset feature extraction network are adjusted, and the step of performing feature extraction on the first sample image using the preset feature extraction network to obtain the first feature data of the first sample image is returned to be executed for each first sample image, until the preset feature extraction network converges and a trained feature extraction network is obtained. The technical scheme provided by the embodiment of the present application improves the performance of the trained feature extraction network.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for feature extraction network training and image recognition. Background Art

[0002] Currently, when processing and analyzing images, a feature extraction network can be used to extract features from the image to obtain feature data, and then image analysis can be performed based on the extracted feature data, such as face recognition, object detection, license plate recognition, etc.

[0003] In related technologies, the aforementioned feature extraction networks are trained using training data. However, due to limitations in privacy protection and data transmission, the acquired training data may be insufficient or even missing for some feature scenarios. Consequently, the performance of the feature extraction networks trained using this training data may be degraded. Summary of the Invention

[0004] The purpose of the embodiments of this application is to provide a method and apparatus for feature extraction network training and image recognition, which generates feature data corresponding to images in various image scenarios during the network training phase, achieves data expansion at the feature level, and thus improves the performance of the trained feature extraction network. The specific technical solution is as follows:

[0005] The present invention provides a method for training a feature extraction network, comprising:

[0006] Acquire a first data set, where the first data set includes a plurality of first sample images and a first true label for each first sample image;

[0007] For each first sample image, using a preset feature extraction network, perform feature extraction on the first sample image to obtain first feature data of the first sample image;

[0008] Inputting the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image to generate second feature data of the first sample image, wherein the preset feature conversion network is trained using feature data of sample images of different image scenes;

[0009] According to the recognition result of the preset recognition network for the first feature data and the second feature data corresponding to each first sample image, and the first true label of each first sample image, the parameters of the preset feature extraction network are adjusted, and the step of performing feature extraction on the first sample image using the preset feature extraction network to obtain the first feature data of the first sample image is returned to be executed until the preset feature extraction network converges to obtain a trained feature extraction network.

[0010] Optionally, the preset feature conversion network is the first generator or the second generator in a pre-trained cyclic adversarial generative network;

[0011] Before inputting the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image to generate the second feature data of the first sample image, the method further includes:

[0012] Acquire a second data set, the second data set including third feature data of a second sample image of the first image scene and fourth feature data of a third sample image of the second image scene;

[0013] Inputting the third feature data of the second sample image into the first generator in the preset recurrent adversarial generative network to obtain fifth feature data corresponding to the second sample image in the second image scene;

[0014] Inputting the fifth feature data corresponding to the second sample image into the second generator in the preset cyclic adversarial generative network to obtain first reconstructed feature data of the second sample image in the first image scene;

[0015] Inputting the fourth feature data corresponding to the third sample image into the second generator to obtain the sixth feature data corresponding to the third sample image in the first image scene;

[0016] inputting the sixth feature data corresponding to the third sample image into the first generator to obtain second reconstructed feature data of the third sample image in the second image scene;

[0017] Calculating a first loss value of the preset cyclic adversarial generative network based on the third feature data, the fourth feature data, the first reconstructed feature data, and the second reconstructed feature data;

[0018] Adjust the parameters of the preset cyclic adversarial generative network based on the first loss value, and return to the step of inputting the third feature data of the second sample image into the first generator in the preset cyclic adversarial generative network to obtain the fifth feature data corresponding to the second sample image in the second image scene, until the preset cyclic adversarial generative network converges to obtain a trained cyclic adversarial generative network.

[0019] Optionally, before performing feature extraction on each first sample image using a preset feature extraction network to obtain first feature data of the first sample image, the method further includes:

[0020] For each target object in a third data set, obtaining a plurality of captured images of the target object in a preset image scene, wherein the third data set is a set of training data for training the preset recognition network;

[0021] For each collected image, extract features of the collected image using the preset feature extraction network to obtain seventh feature data of the collected image;

[0022] An average value of the seventh feature data of the plurality of collected images is calculated, and an initial weight value corresponding to the target object in the preset recognition network is updated.

[0023] Optionally, the step of adjusting the parameters of the preset feature extraction network according to the recognition result of the preset recognition network for the first feature data and the second feature data corresponding to each first sample image, and the first true label of each first sample image, includes:

[0024] Adjusting parameters of a preset recognition network according to a second loss value between the first recognition results, the second recognition results, and the first true label corresponding to the plurality of first sample images to obtain an adjusted preset recognition network, wherein the first recognition result is obtained by the preset recognition network through target recognition based on the first feature data of the first sample image, and the second recognition result is obtained by the preset recognition network through target recognition based on the second feature data of the first sample image;

[0025] According to the third loss value between the third recognition results corresponding to the multiple first sample images and the first true label, the parameters of the preset feature extraction network are adjusted, and the third recognition result is obtained by the adjusted preset recognition network through target recognition based on the first feature data of the first sample image.

[0026] Optionally, the step of adjusting parameters of a preset recognition network according to a second loss value between the first recognition results, the second recognition results, and the first true label corresponding to the plurality of first sample images to obtain an adjusted preset recognition network includes:

[0027] For each first sample image, using a preset recognition network, perform target recognition based on first feature data of the first sample image to obtain a first recognition result of the first sample image, and determine a first predicted label of the first sample image based on the first recognition result;

[0028] Using the preset recognition network, performing target recognition based on the second feature data of the first sample image to obtain a second recognition result of the first sample image, and determining a second predicted label of the first sample image based on the second recognition result;

[0029] Calculating a fourth loss value between the first predicted label and the first true label corresponding to the first sample image;

[0030] Calculating a fifth loss value between the second predicted label and the first true label corresponding to the first sample image;

[0031] Calculating a second loss value of the preset recognition network according to the fourth loss values and the fifth loss values corresponding to the plurality of first sample images;

[0032] Based on the second loss value, the initial weight value corresponding to each target object in the preset recognition network is adjusted, and the step of performing target recognition according to the first feature data of the first sample image using the preset recognition network to obtain a first recognition result of the first sample image is returned to execution for each first sample image, until the preset recognition network converges to obtain an adjusted preset recognition network.

[0033] Optionally, the step of adjusting the parameters of the preset feature extraction network according to a third loss value between the third recognition results corresponding to the plurality of first sample images and the first true label includes:

[0034] For each first sample image, using the adjusted preset recognition network, perform target recognition based on the first feature data of the first sample image to obtain a third recognition result for the first sample image, and determine a third predicted label for the first sample image based on the third recognition result;

[0035] Calculating a loss value between a third predicted label and a first true label corresponding to the plurality of first sample images as a third loss value of the preset feature extraction network;

[0036] Based on the third loss value, parameters of the preset feature extraction network are adjusted.

[0037] Optionally, before obtaining the first data set, the method further includes:

[0038] Acquire a third data set, where the third data set includes a plurality of fourth sample images and a second true label for each fourth sample image;

[0039] For each fourth sample image, extract features of the fourth sample image using the first network model to obtain seventh feature data of the fourth sample image;

[0040] inputting the seventh feature data of the fourth sample image into the second network model to obtain a fourth recognition result of the fourth sample image, and determining a fourth predicted label of the fourth sample image based on the fourth recognition result;

[0041] Calculating a sixth loss value of the first network model and the second network model according to the fourth predicted labels and the second true labels corresponding to the plurality of fourth sample images;

[0042] Based on the sixth loss value, the parameters of the first network model and the second network model are adjusted, and the step of performing feature extraction on each fourth sample image using the first network model to obtain the seventh feature data of the fourth sample image is returned to execution until the first network model and the second network model converge, and the current first network model is determined as the preset feature extraction network, and the current second network model is determined as the preset recognition network.

[0043] The present invention also provides an image recognition method, which includes:

[0044] Obtain the face image to be recognized;

[0045] Inputting the face image to be identified into a feature extraction network to obtain facial feature data of the face image to be identified; wherein the feature extraction network is a feature extraction network trained in any of the above-mentioned feature extraction network training methods;

[0046] Based on the facial feature data, a face recognition result is obtained.

[0047] The present application also provides a feature extraction network training device, comprising:

[0048] A first acquisition module is configured to acquire a first data set, where the first data set includes a plurality of first sample images and a first true label for each first sample image;

[0049] a first extraction module, configured to perform feature extraction on each first sample image using a preset feature extraction network to obtain first feature data of the first sample image;

[0050] a first generating module, configured to input the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image, to generate second feature data of the first sample image, wherein the preset feature conversion network is trained using feature data of sample images of different image scenes;

[0051] The first adjustment module is used to adjust the parameters of the preset feature extraction network according to the recognition results of the first feature data and the second feature data corresponding to each first sample image by the preset recognition network, as well as the first true label of each first sample image, and return to execute the step of performing feature extraction on the first sample image using the preset feature extraction network to obtain the first feature data of the first sample image for each first sample image, until the preset feature extraction network converges to obtain a trained feature extraction network.

[0052] Optionally, the preset feature conversion network is the first generator or the second generator in a pre-trained cyclic adversarial generative network;

[0053] The device further comprises:

[0054] a second acquisition module, configured to acquire a second data set before inputting the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image to generate second feature data of the first sample image, the second data set including third feature data of the second sample image of the first image scene and fourth feature data of the third sample image of the second image scene;

[0055] a second generating module, configured to input the third feature data of the second sample image into the first generator in the preset cyclic adversarial generation network to obtain fifth feature data corresponding to the second sample image in the second image scene;

[0056] a third generating module, configured to input the fifth feature data corresponding to the second sample image into the second generator in the preset cyclic adversarial generative network to obtain first reconstructed feature data of the second sample image in the first image scene;

[0057] a fourth generating module, configured to input the fourth feature data corresponding to the third sample image into the second generator, to obtain sixth feature data corresponding to the third sample image in the first image scene;

[0058] a fifth generating module, configured to input sixth feature data corresponding to the third sample image into the first generator to obtain second reconstructed feature data of the third sample image in the second image scene;

[0059] A first calculation module is configured to calculate a third loss value of the preset cyclic adversarial generative network based on the third feature data, the fourth feature data, the first reconstructed feature data, and the second reconstructed feature data;

[0060] The second adjustment module is used to adjust the parameters of the preset cyclic adversarial generation network based on the third loss value, and return to execute the step of inputting the third feature data of the second sample image into the first generator in the preset cyclic adversarial generation network to obtain the fifth feature data corresponding to the second sample image in the second image scene, until the preset cyclic adversarial generation network converges to obtain a trained cyclic adversarial generation network.

[0061] Optionally, the device further includes:

[0062] a third acquisition module configured to acquire, for each target object in a third data set, a plurality of captured images of the target object in a preset image scene before performing feature extraction on the first sample image using a preset feature extraction network to obtain first feature data of the first sample image, wherein the third data set is a collection of training data for training the preset recognition network;

[0063] a second extraction module, configured to extract features from each collected image using the preset feature extraction network to obtain seventh feature data of the collected image;

[0064] The second calculation module is used to calculate the average value of the seventh feature data of the multiple collected images and update the initial weight value corresponding to the target object in the preset recognition network.

[0065] Optionally, the first adjustment module includes:

[0066] a first adjustment submodule, configured to adjust parameters of a preset recognition network based on a first loss value between a first recognition result, a second recognition result, and a first true label corresponding to the plurality of first sample images, to obtain an adjusted preset recognition network, wherein the first recognition result is obtained by the preset recognition network through target recognition based on the first feature data of the first sample image, and the second recognition result is obtained by the preset recognition network through target recognition based on the second feature data of the first sample image;

[0067] The second adjustment submodule is used to adjust the parameters of the preset feature extraction network according to the second loss value between the third recognition results corresponding to the multiple first sample images and the first true label, and the third recognition result is obtained by the adjusted preset recognition network through target recognition based on the first feature data of the first sample image.

[0068] Optionally, the first adjustment submodule is specifically configured to, for each first sample image, use a preset recognition network to perform target recognition based on first feature data of the first sample image, obtain a first recognition result of the first sample image, and determine a first predicted label of the first sample image based on the first recognition result;

[0069] Using the preset recognition network, performing target recognition based on the second feature data of the first sample image to obtain a second recognition result of the first sample image, and determining a second predicted label of the first sample image based on the second recognition result;

[0070] Calculating a fourth loss value between the first predicted label and the first true label corresponding to the first sample image;

[0071] Calculating a fifth loss value between the second predicted label and the first true label corresponding to the first sample image;

[0072] Calculating a first loss value of the preset recognition network according to the fourth loss values and the fifth loss values corresponding to the plurality of first sample images;

[0073] Based on the first loss value, the initial weight value corresponding to each target object in the preset recognition network is adjusted, and the step of performing target recognition according to the first feature data of the first sample image using the preset recognition network to obtain a first recognition result of the first sample image is returned to execution, until the preset recognition network converges to obtain an adjusted preset recognition network.

[0074] Optionally, the second adjustment submodule is specifically configured to, for each first sample image, use the adjusted preset recognition network to perform target recognition based on the first feature data of the first sample image, obtain a third recognition result of the first sample image, and determine a third predicted label of the first sample image based on the third recognition result;

[0075] Calculating a loss value between a third predicted label and a first true label corresponding to the plurality of first sample images as a second loss value of the preset feature extraction network;

[0076] Based on the second loss value, adjust the parameters of the preset feature extraction network.

[0077] Optionally, the device further includes:

[0078] a fourth acquisition module, configured to acquire a third data set before acquiring the first data set, wherein the third data set includes a plurality of fourth sample images and a second true label for each fourth sample image;

[0079] a third extraction module, configured to perform feature extraction on each fourth sample image using the first network model to obtain seventh feature data of the fourth sample image;

[0080] a first recognition module, configured to input the seventh feature data of the fourth sample image into the second network model to obtain a fourth recognition result of the fourth sample image, and determine a fourth predicted label of the fourth sample image based on the fourth recognition result;

[0081] A third calculation module is used to calculate a sixth loss value of the first network model and the second network model according to the fourth predicted labels and the second true labels corresponding to the plurality of fourth sample images;

[0082] The third adjustment module is used to adjust the parameters of the first network model and the second network model based on the sixth loss value, and return to execute the step of performing feature extraction on each fourth sample image using the first network model to obtain the seventh feature data of the fourth sample image until the first network model and the second network model converge, and determine the current first network model as the preset feature extraction network, and determine the current second network model as the preset recognition network.

[0083] The present application also provides an image recognition device, comprising:

[0084] A fifth acquisition module is used to acquire a face image to be recognized;

[0085] a fourth extraction module, configured to input the face image to be identified into a feature extraction network to obtain facial feature data of the face image to be identified; wherein the feature extraction network is a feature extraction network trained in any of the above-mentioned feature extraction network training methods;

[0086] The recognition module is used to obtain a face recognition result based on the face feature data.

[0087] An embodiment of the present application further provides an electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0088] Memory for storing computer programs;

[0089] The processor is configured to implement any of the above-mentioned steps of the feature extraction network training method when executing the program stored in the memory.

[0090] An embodiment of the present application further provides an electronic device, comprising a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0091] Memory for storing computer programs;

[0092] The processor is configured to implement any of the above-mentioned steps of the image recognition method when executing the program stored in the memory.

[0093] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned steps of the feature extraction network training method is implemented.

[0094] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-mentioned steps of the image recognition method is implemented.

[0095] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-described feature extraction network training methods.

[0096] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the above-described image recognition methods.

[0097] Beneficial effects of the embodiments of the present application:

[0098] The technical solution provided in the embodiment of the present application can use a preset feature extraction network to perform feature extraction on each first sample image in the first data set to obtain first feature data of the first sample image. For each first sample image, the preset feature conversion network corresponding to the image scene of the first sample image is used to generate second feature data of the first sample image in different image scenes, thereby adjusting the parameters of the preset feature extraction network based on the recognition results of the first feature data and the second feature data corresponding to each first sample image by the preset recognition network, as well as the first true label of each first sample image, and repeating the training process of the preset feature extraction network until the preset feature extraction network converges to obtain a trained feature extraction network.

[0099] Through the preset feature conversion network corresponding to the image scene of each first sample image, the second feature data of the first sample image in different image scenes can be generated, that is, the feature data corresponding to the sample image in different image scenes are generated in the network training stage, and the feature data expansion of the feature data of the sample image at the feature level is realized, which effectively improves the applicability of the feature extraction network trained based on the feature data of multiple different image scenes to images of different image scenes, so that the trained feature extraction network can extract features from images collected in different image scenes, thereby improving the performance of the trained feature extraction network.

[0100] In addition, after using the preset feature conversion network to perform feature conversion on the feature data of each sample image to obtain the second feature data, the parameter adjustment process of the preset feature extraction network is affected not only by the first feature data corresponding to each sample image, but also by the second feature data corresponding to each sample image. This makes the expanded second feature data closer to the feature data corresponding to the real image scene, thereby improving the accuracy of the feature extraction network trained based on feature data of multiple different image scenes.

[0101] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0103] Figure 1 A schematic diagram of the first flow chart of the feature extraction network training method provided in an embodiment of the present application;

[0104] Figure 2 A schematic diagram of a recurrent adversarial generative network provided in an embodiment of the present application;

[0105] Figure 3 A second flow chart of the feature extraction network training method provided in an embodiment of the present application;

[0106] Figure 4 A third flow chart of the feature extraction network training method provided in an embodiment of the present application;

[0107] Figure 5 A fourth flow chart of the feature extraction network training method provided in an embodiment of the present application;

[0108] Figure 6A fifth flow chart of the feature extraction network training method provided in an embodiment of the present application;

[0109] Figure 7 A flowchart of a preset identification network parameter adjustment method provided in an embodiment of the present application;

[0110] Figure 8 A schematic diagram of a flow chart of a method for adjusting network parameters for preset feature extraction provided in an embodiment of the present application;

[0111] Figure 9 A sixth flow chart of the feature extraction network training method provided in an embodiment of the present application;

[0112] Figure 10 A schematic diagram of a flow chart of an image recognition method provided in an embodiment of the present application;

[0113] Figure 11 A schematic diagram of the structure of a feature extraction network training device provided in an embodiment of the present application;

[0114] Figure 12 A schematic diagram of the structure of an image recognition device provided in an embodiment of the present application;

[0115] Figure 13 A schematic diagram of the first structure of an electronic device provided in an embodiment of the present application;

[0116] Figure 14 A second structural diagram of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0117] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of this application.

[0118] To address the problems in the related art, the present invention provides a method for training a feature extraction network. This method can be applied to any electronic device, such as a terminal device, a server, etc. The electronic device is not specifically limited herein.

[0119] like Figure 1 As shown, Figure 1 This is a first flow chart of a feature extraction network training method provided in an embodiment of the present application. The method includes the following steps.

[0120] Step S101 : Acquire a first data set, where the first data set includes a plurality of first sample images and a first true label for each first sample image.

[0121] In step S102 , for each first sample image, a preset feature extraction network is used to perform feature extraction on the first sample image to obtain first feature data of the first sample image.

[0122] In step S103, the first feature data of the first sample image is input into a preset feature conversion network corresponding to the image scene of the first sample image to generate second feature data of the first sample image. The preset feature conversion network is trained using feature data of sample images of different image scenes.

[0123] Step S104: Adjust the parameters of the preset feature extraction network according to the recognition results of the preset recognition network for the first feature data and the second feature data corresponding to each first sample image, as well as the first true label of each first sample image, and return to the step of performing feature extraction on each first sample image using the preset feature extraction network to obtain the first feature data of the first sample image, until the preset feature extraction network converges to obtain a trained feature extraction network.

[0124] pass Figure 1 The method shown can use a preset feature extraction network to perform feature extraction on each first sample image in the first data set to obtain first feature data of the first sample image. For each first sample image, the preset feature conversion network corresponding to the image scene of the first sample image is used to generate second feature data of the first sample image in different image scenes, thereby adjusting the parameters of the preset feature extraction network based on the recognition results of the first feature data and the second feature data corresponding to each first sample image by the preset recognition network, as well as the first true label of each first sample image, and repeating the training process of the preset feature extraction network until the preset feature extraction network converges to obtain a trained feature extraction network.

[0125] Through the preset feature conversion network corresponding to the image scene of each first sample image, the second feature data of the first sample image in different image scenes can be generated, that is, the feature data corresponding to the sample image in different image scenes are generated in the network training stage, and the feature data expansion of the feature data of the sample image at the feature level is realized, which effectively improves the applicability of the feature extraction network trained based on the feature data of multiple different image scenes to images of different image scenes, so that the trained feature extraction network can extract features from images collected in different image scenes, thereby improving the performance of the trained feature extraction network.

[0126] In addition, after using the preset feature conversion network to perform feature conversion on the feature data of each sample image to obtain the second feature data, the parameter adjustment process of the preset feature extraction network is affected not only by the first feature data corresponding to each sample image, but also by the second feature data corresponding to each sample image. This makes the expanded second feature data closer to the feature data corresponding to the real image scene, thereby improving the accuracy of the feature extraction network trained based on feature data of multiple different image scenes.

[0127] The embodiments of the present application are described below through specific examples.

[0128] Regarding the above step S101 , that is, obtaining a first data set, the first data set includes a plurality of first sample images and a first true label of each first sample image.

[0129] In this step, the electronic device can obtain multiple sample images (referred to as first sample images) and the label of the target object in each first sample image (referred to as the first real label) to obtain a training data set (referred to as the first data set).

[0130] The objects in the first sample images include, but are not limited to, faces, license plates, and the like. The first true label of each first sample image may be used to indicate attribute information of the object in the first sample image, such as the identity information of the person corresponding to a face, the license plate number corresponding to a license plate, and the like. The objects in the first sample images and the first true label of each first sample image are not specifically limited herein.

[0131] For ease of understanding, the following description only uses the example of a sample image being a face image, a target object being a face, and the true label being the identity information of the person corresponding to the face, and does not serve any limiting purpose.

[0132] In the embodiment of the present application, there is no specific limitation on the number of first sample images included in the first data set.

[0133] In an optional embodiment, the first sample image may be a pre-processed sample image.

[0134] For example, when the first sample images are facial images, each of the first sample images is a facial image that has undergone normalization processing, that is, the first sample images are facial images that have undergone face detection and face positioning.

[0135] The above face detection process can be completed using machine learning methods such as boosting algorithm, Fast Region-based Convolutional Network (fast RCNN) algorithm, Faster Region-based Convolutional Network (faster RCNN) algorithm, you only look once (Yolo), Single Shot MultiBox Detector (SSD) algorithm, or manual labeling.

[0136] The above-mentioned face alignment can also use machine learning methods or manual labeling to obtain the coordinates of the key points in the face image, such as 3 points, 5 points or 68 points in the related technology, and then calculate the radial transformation matrix according to a specific template. According to the coordinates of the key points and the radial transformation matrix, the face image is transformed into a standard template to obtain the face image after face alignment.

[0137] In the embodiment of the present application, the pre-processing method of the sample image is different according to the type of the target in the first sample image. Here, the first pre-processing method is not specifically limited.

[0138] With respect to the above step S102 , that is, with respect to each first sample image, a preset feature extraction network is used to perform feature extraction on the first sample image to obtain first feature data of the first sample image.

[0139] In this step, the electronic device may obtain a pre-trained preset feature extraction network for extracting image feature data. For each first sample image in the first dataset, the electronic device may use the pre-trained feature extraction network to extract features of the target object in the first sample image, thereby obtaining feature data for the first sample image (referred to as first feature data).

[0140] For example, the electronic device can use the above-mentioned preset feature extraction network to perform feature extraction on the above-mentioned facial image to obtain facial feature data of the target face in the facial image.

[0141] In an embodiment of the present application, the preset feature extraction network can be obtained by training the first network model using a preset training set. The preset training set can be the first data set described above, or another data set. The training method for the first network model can be described below and will not be repeated here.

[0142] The first network model can be a deep learning network model. That is, the preset feature extraction network can be a deep learning network model, such as a convolutional neural network model. The network structures of the first network model and the preset feature extraction network are not specifically limited.

[0143] In an embodiment of the present application, when the electronic device uses the preset feature extraction network to extract features from the first sample image, it can obtain feature data output by any intermediate layer in the preset feature extraction network as the first feature data.

[0144] For ease of understanding, the following description is made using a convolutional neural network model as an example. Assume that the preset feature extraction network includes multiple convolutional pooling layers, such as five convolutional pooling layers. The electronic device can obtain the output data of any convolutional pooling layer as the first feature data.

[0145] For example, the electronic device can obtain the output result of the first convolution pooling layer in the preset feature extraction network as the first feature data, or can obtain the output result of the fifth convolution pooling layer in the preset feature extraction network as the first feature data.

[0146] In an optional embodiment, in order to improve the accuracy of the first feature data extracted above, the electronic device can obtain the output features of the above-mentioned preset feature extraction network as the first feature data when obtaining the above-mentioned first feature data, that is, obtain the output data of the last layer in the preset feature extraction network as the first feature data.

[0147] For the above-mentioned step S103, the first feature data of the first sample image is input into the preset feature conversion network corresponding to the image scene of the first sample image to generate the second feature data of the first sample image. The preset feature conversion network is trained using the feature data of sample images of different image scenes.

[0148] In an embodiment of the present application, the preset feature conversion network can be used to convert feature data of an image of a certain image scene into feature data corresponding to another image scene. The image scenes corresponding to the feature data before and after conversion by the preset feature conversion network are different.

[0149] For ease of understanding, the above-mentioned facial image is still used as an example for explanation. Assume that the facial image is a facial image of a person captured in a first image scene, such as a frontal image of the person. After the electronic device extracts the feature data of the facial image, it can input the feature data into the above-mentioned preset feature conversion network. In this case, the preset feature conversion network will output the facial feature data corresponding to the person in the second image scene, such as the feature data of the person's profile image. The first image scene is different from the second image scene.

[0150] In an embodiment of the present application, the image scenes corresponding to the feature data before conversion and the feature data after conversion by the above-mentioned preset feature conversion module are determined according to the image scenes of the sample images corresponding to the feature data participating in the training during the training of the preset feature conversion network. Here, no specific limitation is made to the above-mentioned first image scene and second image scene. For ease of understanding, the following description will only take the first image scene as a facial image captured in a scene where a person's front face can be captured, and the second image scene as a side face image captured in a scene where a person's side face can be captured as an example, which does not serve any limiting purpose.

[0151] In an optional embodiment, the above-mentioned preset feature conversion network can be the first generator or the second generator in a pre-trained cyclic adversarial generation network.

[0152] The first generator may be used to convert feature data of the image of the first image scene into feature data in the second image scene. The second generator may be used to convert feature data of the image of the second image scene into feature data in the first image scene.

[0153] For ease of understanding, combined Figure 2 Take this as an example to illustrate: Figure 2 A schematic diagram of a recurrent adversarial generative network provided in an embodiment of the present application.

[0154] exist Figure 2 In the cyclic adversarial generation network shown in FIG, X is the feature data of the image of the first image scene, Y is the feature data of the second image scene, G is the first generator, F is the second generator, and D X is the first discriminator, D Y is the second discriminator. Among them, the first discriminator D X The second discriminator D is used to determine whether the input feature data is the feature data of the image of the first image scene. Y Used to determine whether the input feature data is the feature data of the image of the second image scene

[0155] The feature data X can be converted into feature data corresponding to the second image scene through the first generator G; the feature data Y can be converted into feature data corresponding to the first image scene through the second generator F.

[0156] The training process of the above-mentioned cyclic adversarial generative network can be found in the description below and will not be explained in detail here.

[0157] In the embodiment of the present application, the first sample image may include a sample image of the first image scene, and may also include a sample image of the second image scene.

[0158] In an optional embodiment, with respect to step S102 above, the first feature data of the first sample image is input into a preset feature conversion network corresponding to the image scene of the first sample image to generate second feature data of the first sample image, which can be specifically expressed as:

[0159] For each first sample image, if the first sample image is a sample image of a first image scene, the first feature data of the first sample image is input into the first generator in the above-mentioned cyclic adversarial generation network to obtain the second feature data corresponding to the first sample image in the second image scene; if the first sample image is a sample image in the second image scene, the first feature data of the first sample image is input into the second generator in the above-mentioned cyclic adversarial generation network to obtain the second feature data of the first sample image in the first image scene.

[0160] In addition to being the first generator or the second generator in the recurrent generative adversarial network, the preset feature conversion network may also be multiple independent feature conversion networks, such as multiple generative adversarial networks. The preset feature conversion network is not specifically limited herein.

[0161] In the above embodiment, only the feature data conversion of two image scenes, namely the first image scene and the second image scene, is used as an example for explanation. In addition, the above preset feature conversion network can also be used to convert feature data of multiple different image scenes.

[0162] For example, the preset feature conversion network can convert the feature data of the images of the first image scene, the second image scene and the third image scene. Figure 2 Take the cyclic adversarial generation network shown in the figure as an example. In this case, the number of generators and discriminators in the cyclic adversarial generation network will increase accordingly. Figure 2 The cyclic adversarial generative network shown in the figure adds a generator and a discriminator for converting between the first image scene and the third image scene, and / or a generator and a discriminator for converting between the second image scene and the third image scene. Figure 2 The recurrent adversarial network shown is used as an example for illustration and does not serve any limiting purpose.

[0163] The conversion between feature data of images of various different image scenes can refer to the above-mentioned feature data conversion method, which will not be described in detail here. For ease of understanding, the following description only uses the conversion between feature data of images of two image scenes as an example, which does not serve any limiting purpose.

[0164] For the above step S104, that is, according to the recognition results of the preset recognition network for the first feature data and the second feature data corresponding to each first sample image, and the first true label of each first sample image, the parameters of the preset feature extraction network are adjusted, and the step of performing feature extraction on the first sample image using the preset feature extraction network to obtain the first feature data of the first sample image is returned to be executed for each first sample image, until the preset feature extraction network converges and a trained feature extraction network is obtained.

[0165] In this step, the electronic device can use the preset recognition network to perform target recognition based on the first feature data and the second feature data corresponding to each sample image, and obtain corresponding recognition results. Based on the recognition results obtained by target recognition and the first true label of each first sample image, the electronic device can calculate the loss value to obtain the loss value of the preset feature extraction network, and based on the loss value, adjust the parameters of the preset feature recognition network, and return to execute the above-mentioned step S102, that is, return to execute the step of extracting features from each first sample image using the preset feature extraction network to obtain the first feature data of the first sample image, until the preset feature extraction network converges, and the preset feature extraction network at the current moment is determined as the trained feature extraction network, completing the training process of the feature extraction network.

[0166] The parameters of the preset feature extraction network adjusted by the electronic device include but are not limited to the weights and biases in the preset feature extraction network. The parameter adjustment method of the preset feature extraction network can be found in the following description and will not be described in detail here.

[0167] In an optional embodiment, when the preset feature conversion network is the first generator or the second generator in a pre-trained cyclic adversarial generation network, according to Figure 1 The method shown in the embodiment of the present application also provides a feature extraction network training method. Figure 3 As shown, Figure 3 This is a second flow chart of the feature extraction network training method provided in the embodiment of the present application. The method includes the following steps.

[0168] Step S301: Acquire a first data set, where the first data set includes a plurality of first sample images and a first true label for each first sample image.

[0169] In step S302 , for each first sample image, a preset feature extraction network is used to perform feature extraction on the first sample image to obtain first feature data of the first sample image.

[0170] The above steps S301 and S302 are the same as the above steps S101 and S102.

[0171] Step S303 : Acquire a second data set, where the second data set includes third feature data of the second sample image of the first image scene and fourth feature data of the third sample image of the second image scene.

[0172] The number of the third feature data included in the second data set may be multiple, and the number of the fourth feature data included in the second data set may also be multiple. Here, the number of the third feature data and the fourth feature data is not specifically limited.

[0173] The method for obtaining the third characteristic data and the fourth characteristic data may refer to the method for obtaining the first characteristic data, and will not be described in detail here.

[0174] In the embodiments of the present application, the training process of the preset feature conversion network is described using only the second dataset including feature data, namely the third and fourth feature data. Alternatively, the data included in the second dataset may be sample images. In this case, the second dataset may be the same as or different from the first dataset.

[0175] In an optional embodiment, when the data included in the second dataset is sample images, the electronic device may utilize the preset feature extraction network to perform feature extraction on each sample image in the second dataset before executing step S304 to obtain feature data for each sample image. The method for extracting this feature data may refer to the method for extracting the first feature data described above and is not described in detail here.

[0176] Step S304: input the third feature data of the second sample image into the first generator in the preset cyclic adversarial generation network to obtain the fifth feature data corresponding to the second sample image in the second image scene.

[0177] Step S305: input the fifth feature data corresponding to the second sample image into the second generator in the preset cyclic adversarial generation network to obtain first reconstructed feature data of the second sample image in the first image scene.

[0178] For ease of understanding, combined with the above Figure 2 The above-mentioned steps S304 and S305 are described.

[0179] For each third feature data of the second sample image, the electronic device may use the third feature data of the second sample image as Figure 2 The feature data X shown is input into the first generator G, and the first generator G will output the feature data of the second sample image in the second image scene (recorded as the fifth feature data).

[0180] The electronic device can use the fifth feature data output by the first generator G as Figure 2 The feature data Y shown is output to the second generator F, and the second generator F will output the reconstructed feature data of the second sample image in the first image scene (referred to as first reconstructed feature data).

[0181] Step S306: input the fourth feature data corresponding to the third sample image into the second generator to obtain the sixth feature data corresponding to the third sample image in the first image scene.

[0182] The method for obtaining the sixth characteristic data may refer to the method for obtaining the first reconstructed characteristic data, and will not be described in detail here.

[0183] Step S307: input the sixth feature data corresponding to the third sample image into the first generator to obtain second reconstructed feature data of the third sample image in the second image scene.

[0184] The method for obtaining the second reconstructed feature data may refer to the method for obtaining the fifth feature data, and will not be described in detail here.

[0185] In the embodiment of the present application, the execution order of the above steps S304-S305 and the above steps S306-S307 is not specifically limited.

[0186] Step S308: Calculate a first loss value of a preset cyclic adversarial generative network based on the third feature data, the fourth feature data, the first reconstructed feature data, and the second reconstructed feature data.

[0187] In an optional embodiment, the electronic device may use the following formula to calculate the loss value (denoted as the first loss value) of the above-mentioned preset cyclic adversarial generation network.

[0188] L(G,F,D X ,D Y )=L GAN (G,D Y ,X,Y)+L GAN (F,D X ,Y,X)+αL cyc (G,F)+βL cls

[0189] Among them, L(G,F,D X ,D Y ) is about the first generator G, the second generator F, and the first discriminator D X and the second discriminator D Y The overall loss function, L GAN (G,D Y ,X,Y) is about the first generator G and the discriminator DY , the loss function between feature data X and feature data Y, GAN represents the name of the loss function, x and y are input data, is the expectation on the distribution of feature data Y, D Y () is the output of the second discriminator when the content in the brackets is input data, log is the logarithmic operation, is the expectation on the distribution of feature data X, G() is the output of the first generator when the content in brackets is input data, L GAN (F,D X ,Y,X) is about the second generator F and the discriminator D X , the loss function between feature data Y and feature data XY, D X () is the output of the first discriminator when the content in the brackets is input data, α is the balance coefficient, L cyc (G,F) is the reconstruction loss function, || ||1 is the L1 norm, F() is the output of the first generator when the content in the brackets is input data, β is the balance coefficient, L cls is the classification loss function (such as cross entropy loss function, etc.), X∪Y is the union of all input feature data, The predicted value output by the preset recognition network.

[0190] For the above-mentioned input data x and y, the above-mentioned input data x and y will also be different depending on the generator or discriminator to which the input data is input. For ease of understanding, taking the above-mentioned step S304 as an example, the fifth feature data is input to the second generator to obtain the first reconstructed feature data. In this case, the input data x of the first generator G is the above-mentioned fifth feature data, and G(x) is the above-mentioned first reconstructed feature data. Here, the inputs and outputs of the above-mentioned first generator, first discriminator, second generator, and second discriminator are not specifically described.

[0191] The L1 norm can be understood as a distance metric. In the above embodiment, only the L1 norm is used as an example for description. In addition, other norms such as the L2 norm can also be used.

[0192] In the embodiment of the present application, the feature data X and feature data Y are different according to the different generators into which the feature data are input during the calculation of the first loss value. For example, for the above steps S304 and S305, L GAN (G,D Y,X,Y) can be the third feature data, and the feature data Y can be the fifth feature data. For another example, with respect to the above steps S306 and S307, L GAN (F,D X ,Y,X) can be the fourth feature data, and the feature data Y can be the sixth feature data.

[0193] Step S309: Adjust the parameters of the preset cyclic adversarial generation network based on the first loss value, and return to the step of inputting the third feature data of the second sample image into the first generator in the preset cyclic adversarial generation network to obtain the fifth feature data corresponding to the second sample image in the second image scene, until the preset cyclic adversarial generation network converges to obtain a trained cyclic adversarial generation network.

[0194] In an embodiment of the present application, after determining the first loss value of the above-mentioned preset cyclic adversarial generation network, the electronic device can determine whether the preset cyclic adversarial generation network has converged based on the first loss value and / or the number of training times of the preset cyclic adversarial generation network.

[0195] If the preset cyclic adversarial network does not converge, the electronic device can adjust the parameters of the preset cyclic adversarial network based on the above-mentioned first loss value, and return to execute the above-mentioned step S304, that is, continue to train the preset cyclic adversarial network until the preset cyclic adversarial network converges.

[0196] If the preset cyclic adversarial generative network converges, the electronic device may determine the current preset cyclic adversarial generative network as the trained cyclic adversarial generative network. In other words, the electronic device may determine the first generator and the second generator in the current preset cyclic adversarial generative network as the trained preset feature conversion network.

[0197] In the above-mentioned recurrent adversarial generative network, the first discriminator and the second discriminator are used to force the first generator and the second generator to generate feature data that is closer to the feature data of the image collected in the real image scene, effectively ensuring the accuracy of the feature data obtained by feature level expansion, thereby ensuring the retention of the target object attribute information in the feature data obtained by feature conversion.

[0198] Through the above steps S303 to S309, the electronic device can use the above second data set to train the above preset feature conversion network, effectively ensuring the accuracy of the trained preset feature conversion network, thereby ensuring the accuracy of the feature data obtained by using the preset feature conversion network to perform feature-level data expansion.

[0199] In step S310, the first feature data of the first sample image is input into a preset feature conversion network corresponding to the image scene of the first sample image to generate second feature data of the first sample image. The preset feature conversion network is trained using feature data of sample images of different image scenes.

[0200] In this step, for each first sample image, the electronic device inputs the first feature data of the first sample image into the first generator or the second generator corresponding to the image scene of the first sample image (i.e., the first generator or the second generator in the cyclic adversarial generative network trained in the above step S309) to generate the second feature data of the first sample image.

[0201] Step S311: Adjust the parameters of the preset feature extraction network according to the recognition results of the preset recognition network for the first feature data and the second feature data corresponding to each first sample image, as well as the first true label of each first sample image, and return to the step of performing feature extraction on each first sample image using the preset feature extraction network to obtain the first feature data of the first sample image, until the preset feature extraction network converges to obtain a trained feature extraction network.

[0202] The above step S311 is the same as the above step S104.

[0203] In an optional embodiment, according to the above Figure 1 The method shown in the embodiment of the present application also provides a feature extraction network training method. Figure 4 As shown, Figure 4 This is a third flow chart of the feature extraction network training method provided in an embodiment of the present application. The method includes the following steps.

[0204] Step S401: Acquire a first data set, where the first data set includes a plurality of first sample images and a first true label for each first sample image.

[0205] The above step S401 is the same as the above step S101.

[0206] Step S402 : For each target object in the third data set, a plurality of captured images of the target object in a preset image scene are obtained. The third data set is a set of training data for training a preset recognition network.

[0207] In the embodiment of the present application, since the above-mentioned preset recognition network is obtained by training with data in the third data set, the number of target objects that can be recognized by the trained preset recognition network corresponds to the number of target objects included in the third data set.

[0208] Taking the third data set as an example of sample images, when the number of people corresponding to the sample images in the third data set is 1000, the people that can be recognized by the preset recognition network trained by the third data set are these 1000 people.

[0209] The training data in the third dataset may be the sample images. The third dataset also includes a true label (referred to as a second true label) for each sample image.

[0210] In an optional embodiment, for each target object in the third data set, when the electronic device obtains multiple captured images of the target object in a preset image scene, the electronic device can obtain all captured images whose image scene is the preset image scene from a pre-stored image set of the target object.

[0211] In the embodiment of the present application, there is no specific limitation on the method for acquiring the above-mentioned multiple collected images.

[0212] The number of the above-mentioned collected images can be set according to user needs or user experience. In addition, the number of collected images of each target object can be the same or different. Here, the number of the above-mentioned collected images is not specifically limited.

[0213] The above-mentioned preset image scenes can be set according to user needs, etc., and the above-mentioned preset image scenes are not specifically limited here.

[0214] In an optional embodiment, to ensure that each captured image includes more image information and thus ensures the comprehensiveness of the feature data extracted based on the captured image, the preset image scene may be an image scene that includes more target attribute information. For example, using a facial image as an example, to ensure that the captured image includes more facial attribute information, the image scene corresponding to an ID photo may be determined as the preset image scene.

[0215] In an optional embodiment, when the number of captured images of the target object in a preset image scene is relatively small or even none, the electronic device may acquire other image scenes, such as multiple captured images in the first image scene or the second image scene.

[0216] Step S403 : For each collected image, a preset feature extraction network is used to perform feature extraction on the collected image to obtain seventh feature data of the collected image.

[0217] The method for extracting the seventh feature data of the above-mentioned captured image may refer to the method for extracting the first feature data, and will not be described in detail here.

[0218] Step S404 , calculating an average value of the seventh feature data of the plurality of collected images, and updating an initial weight value corresponding to the target object in a preset recognition network.

[0219] In an optional embodiment, if the number of images of each target object acquired through step S402 is N, the seventh feature data of each target object extracted through step S403 is expressed as: {f i,j}(i=1,2,…,N), then the average value of the seventh feature data of the above multiple collected images can be expressed as:

[0220]

[0221] Among them, w i is the average value of the seventh feature data of multiple collected images of the i-th target object.

[0222] For each target object, after calculating the average value of the seventh feature data of multiple captured images of the target object, the electronic device can use the average value to update the initial weight value corresponding to the target object in the preset recognition network. In other words, the average value of the seventh feature data corresponding to each target object in the preset recognition network obtained through training is updated as the initial weight value of the target object.

[0223] In an embodiment of the present application, by updating the initial weight value of each target object in the above-mentioned preset recognition network to the average value of the seventh characteristic data of the target object, the weight of the target object in the preset recognition network can be located near the convergence area, which can effectively shorten the training time of the preset recognition network, thereby improving the training efficiency of the preset recognition network.

[0224] Step S405 : For each first sample image, a preset feature extraction network is used to perform feature extraction on the first sample image to obtain first feature data of the first sample image.

[0225] Step S406: input the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image to generate second feature data of the first sample image. The preset feature conversion network is trained using feature data of sample images of different image scenes.

[0226] Step S407: Adjust the parameters of the preset feature extraction network according to the recognition results of the preset recognition network for the first feature data and the second feature data corresponding to each first sample image, as well as the first true label of each first sample image, and return to the step of performing feature extraction on each first sample image using the preset feature extraction network to obtain the first feature data of the first sample image, until the preset feature extraction network converges to obtain a trained feature extraction network.

[0227] The above steps S405 to S407 are the same as the above steps S102 to S104.

[0228] In an optional embodiment, according to Figure 1 The method shown in the embodiment of the present application also provides a feature extraction network training method. Figure 5 As shown, Figure 5 This is a fourth flow chart of the feature extraction network training method provided in an embodiment of the present application. The method includes the following steps.

[0229] Step S501 : Acquire a first data set, where the first data set includes a plurality of first sample images and a first true label for each first sample image.

[0230] In step S502 , for each first sample image, a preset feature extraction network is used to perform feature extraction on the first sample image to obtain first feature data of the first sample image.

[0231] Step S503: input the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image to generate second feature data of the first sample image. The preset feature conversion network is trained using feature data of sample images of different image scenes.

[0232] The above steps S501 to S503 are the same as the above steps S101 to S103.

[0233] Step S504: Adjust the parameters of the preset feature extraction network according to the second loss value between the first recognition results, the second recognition results, and the first true label corresponding to the plurality of first sample images, and return to the step of performing feature extraction on each first sample image using the preset feature extraction network to obtain the first feature data of the first sample image, until the preset feature extraction network converges and a trained feature extraction network is obtained; wherein the first recognition result is obtained by the preset recognition network performing target recognition based on the first feature data of the first sample image, and the second recognition result is obtained by the preset recognition network performing target recognition on the second feature data of the first sample image.

[0234] In this step, the electronic device may perform target recognition on the first feature data and the second feature data of each first sample image according to the preset recognition network to obtain the first recognition result and the second recognition result, as well as the first true label of each first sample image, and calculate the loss value (recorded as the second loss value) between the first recognition result, the second recognition result, and the first true label as the loss value of the preset feature extraction network. The electronic device may adjust the parameters of the preset feature extraction network based on the second loss value and return to execute the above step S502 until the preset feature extraction network converges to obtain a trained feature extraction network.

[0235] The second loss value is calculated based on the loss between the first recognition result and the first true label for each first sample image, and the loss between the second recognition result and the first true label for each sample image. The calculation of the second loss value is described below and is not detailed here.

[0236] The above step S504 is a refinement of the above step S104.

[0237] In an embodiment of the present application, since the second loss value is calculated based on the loss value between the first recognition result and the first true label corresponding to each first sample image, and the loss value between the second recognition result and the first true label corresponding to each sample image, the smaller the second loss value, the higher the accuracy of the first recognition result and the second recognition result, which effectively improves the accuracy of the feature extraction network trained based on the second feature data.

[0238] In another optional embodiment, according to Figure 1 The method shown in the embodiment of the present application also provides a feature extraction network training method. Figure 6 As shown, Figure 6 This is a fifth flow chart of the feature extraction network training method provided in the embodiment of the present application. The method includes the following steps.

[0239] Step S601: Acquire a first data set, where the first data set includes a plurality of first sample images and a first true label for each first sample image.

[0240] In step S602 , for each first sample image, a preset feature extraction network is used to perform feature extraction on the first sample image to obtain first feature data of the first sample image.

[0241] Step S603: input the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image to generate second feature data of the first sample image. The preset feature conversion network is trained using feature data of sample images of different image scenes.

[0242] The above steps S601 to S603 are the same as the above steps S101 to S103.

[0243] Step S604: Adjust the parameters of the preset recognition network according to the first recognition results, the second recognition results, and the second loss value between the first true label corresponding to multiple first sample images to obtain an adjusted preset recognition network. The first recognition result is obtained by the preset recognition network through target recognition based on the first feature data of the first sample image, and the second recognition result is obtained by the preset recognition network through target recognition based on the second feature data of the first sample image.

[0244] In this step, after the electronic device obtains the second feature data corresponding to each first sample image using the above-mentioned preset feature conversion network, it can use the preset recognition network to perform target recognition based on the first feature data and the second feature data of each sample image respectively, to obtain a first recognition result and a second recognition result, thereby calculating the second loss value of the preset recognition network based on the difference between the first recognition result corresponding to each first sample image and the first true label, and the difference between the second recognition result corresponding to each first sample image and the first true label, and fine-tuning the parameters of the preset recognition network based on the second loss value to obtain an adjusted preset recognition network.

[0245] In this embodiment of the present application, the preset recognition network includes parameters for multiple objects, each of which is used to indicate a weight for that object. The recognition result of the preset recognition network is the predicted value output by the preset recognition network. The recognition result can be represented as a recognition array, where each array element in the recognition array indicates the probability that the object in the first sample image is a respective object in the preset recognition network.

[0246] The multiple targets in the above-mentioned preset recognition network are targets identified by the preset recognition network. According to the different targets in the sample data used to train the preset recognition network, the targets represented by the preset recognition network are also different.

[0247] The preset recognition network can be trained together with the preset feature extraction network. The training process of the preset recognition network can be found in the following description and will not be described in detail here.

[0248] In an optional embodiment, Figure 7 As shown, Figure 7 A flow chart of a preset identification network parameter adjustment method provided in an embodiment of the present application. The method includes the following steps.

[0249] In step S701 , for each first sample image, a preset recognition network is used to perform target recognition based on first feature data of the first sample image to obtain a first recognition result of the first sample image, and a first predicted label of the first sample image is determined based on the first recognition result.

[0250] In an optional embodiment, when the above-mentioned recognition result can be expressed as a recognition array, the electronic device determines the first predicted label of the first sample image based on the first recognition result. Specifically, it can be expressed as: the electronic device can determine the attribute information of the target object corresponding to the largest array element in the recognition array as the first predicted label of the first sample image.

[0251] In another optional embodiment, the above recognition result may include a corresponding predicted label, and the electronic device may directly determine the predicted label included in the first recognition result as the first predicted label of the first sample image.

[0252] Here, the method for determining the first prediction label is not specifically limited.

[0253] Step S702 : Using a preset recognition network, perform target recognition based on the second feature data of the first sample image to obtain a second recognition result of the first sample image, and determine a second predicted label of the first sample image based on the second recognition result.

[0254] The method for determining the second recognition result can refer to the method for determining the first recognition result, and the method for determining the second predicted label can refer to the method for determining the first predicted label. Here, the process of determining the second recognition result and the second predicted label is not described in detail.

[0255] Step S703: Calculate a fourth loss value between the first predicted label and the first true label corresponding to the first sample image.

[0256] In this step, for each first sample image, the electronic device may calculate a loss value (referred to as a fourth loss value) between the first predicted label and the first true label of the first sample image using a preset loss function, such as a cross-entropy loss function. The preset loss function is not specifically limited herein.

[0257] Step S704 , calculating a fifth loss value between the second predicted label and the first true label corresponding to the first sample image.

[0258] The calculation method of the fifth loss value can refer to the calculation method of the fourth loss value, which will not be described in detail here.

[0259] Step S705 : Calculate a second loss value of the preset recognition network according to the fourth loss values and the fifth loss values corresponding to the plurality of first sample images.

[0260] In an optional embodiment, when calculating the above-mentioned second loss value, the electronic device may calculate the sum of the fourth loss value and the fifth loss value corresponding to each first sample image as the second loss value of the preset recognition network.

[0261] In another optional embodiment, when calculating the above-mentioned second loss value, the electronic device may calculate a weighted sum of the fourth loss value and the fifth loss value corresponding to each first sample image as the second loss value of the preset recognition network.

[0262] In the embodiment of the present application, there is no specific limitation on the calculation method of the second loss value.

[0263] Step S706: Adjust the initial weight value corresponding to each target object in the preset recognition network based on the second loss value, and return to the step of performing target recognition for each first sample image using the preset recognition network according to the first feature data of the first sample image to obtain the first recognition result of the first sample image, until the preset recognition network converges to obtain the adjusted preset recognition network.

[0264] In an embodiment of the present application, after calculating the second loss value, the electronic device can determine whether the preset recognition network has converged based on the second loss value and / or the number of training times of the preset recognition network.

[0265] If the preset recognition network does not converge, the electronic device may adjust the weight corresponding to each target object in the preset recognition network based on the second loss value, that is, adjust the updated initial weight value until the preset recognition network converges.

[0266] If the preset recognition network converges, the electronic device may determine the current preset recognition network as the adjusted preset recognition network.

[0267] In the embodiment of the present application, when the electronic device adjusts the initial weight value based on the second loss value, it can fine-tune the initial weight value, that is, adjust the initial weight value according to a smaller adjustment step.

[0268] Through the above steps S701 to S706, the electronic device can accurately calculate the second loss value of the preset recognition network, thereby completing the parameter adjustment of the preset recognition network based on the second loss value, thereby improving the accuracy of the preset recognition network parameter adjustment.

[0269] Step S605: Adjust the parameters of the preset feature extraction network according to the third loss value between the third recognition results corresponding to the multiple first sample images and the first true label, and return to the step of performing feature extraction on each first sample image using the preset feature extraction network to obtain the first feature data of the first sample image, until the preset feature extraction network converges and a trained feature extraction network is obtained; the third recognition result is obtained by the adjusted preset recognition network performing target recognition based on the first feature data of the first sample image.

[0270] In an optional embodiment, the step S605 of adjusting the parameters of the preset feature extraction network according to the third loss value between the third recognition results corresponding to the plurality of first sample images and the first true label may include the following steps: Figure 8 As shown, Figure 8 A flow chart of a method for adjusting network parameters for extracting preset features provided in an embodiment of the present application. The method includes the following steps.

[0271] In step S801 , for each first sample image, the adjusted preset recognition network is used to perform target recognition based on the first feature data of the first sample image, thereby obtaining a third recognition result of the first sample image, and determining a third predicted label of the first sample image based on the third recognition result.

[0272] In this step, after obtaining the adjusted preset recognition network through the above-mentioned step S706, for each of the above-mentioned first sample images, the electronic device can input the first feature data of the first sample image into the adjusted preset recognition network to obtain the recognition result of the target object in the first sample image (recorded as the third recognition result), and determine the predicted label of the first sample image (recorded as the third predicted label) based on the third recognition result.

[0273] The method for determining the third recognition result can refer to the method for determining the first recognition result, and the method for determining the third predicted label can refer to the method for determining the first predicted label. Here, the process of determining the third recognition result and the third predicted label is not described in detail.

[0274] Step S802: Calculate the loss value between the third predicted labels and the first true labels corresponding to the plurality of first sample images as the third loss value of the preset feature extraction network.

[0275] In this step, after determining the third predicted label corresponding to each first sample image, the electronic device may calculate a loss value (referred to as a third loss value) between the third predicted label and the first true label corresponding to each first sample image using a preset loss function, such as a cross-entropy loss function, as the third loss value of the preset feature extraction network. The preset loss function is not specifically limited herein.

[0276] In the above steps S801 and S802, since the output result of the above-mentioned preset feature extraction network is only the above-mentioned first feature data, and the above-mentioned adjusted preset recognition network is parameter-adjusted by the second loss value calculated by the above-mentioned fourth loss value and the fifth loss value, therefore, through the above-mentioned adjusted preset recognition network, while realizing data expansion of the feature data of the sample image at the feature level, the accuracy of the preset recognition network after parameter adjustment based on the second feature data is improved, thereby improving the accuracy of the third recognition result identified by the adjusted preset recognition network, as well as the accuracy of the loss value corresponding to the preset feature extraction network determined based on the third predicted label and the first true label, which facilitates the subsequent parameter adjustment process of the preset feature extraction network, realizes the training of the preset feature extraction network, and improves the adaptability of the trained feature extraction network to feature extraction of images of different image scenes.

[0277] In the above embodiment, both step S801 and step S802 are performed by the electronic device. Alternatively, the third predicted label and the third loss value may be obtained by processing the preset recognition model. That is, the output result of the preset recognition model includes the third predicted label and the third loss value.

[0278] Step S803: Adjust the parameters of the preset feature extraction network based on the third loss value.

[0279] In an embodiment of the present application, after determining the third loss value of the above-mentioned preset feature extraction network, the electronic device can determine whether the preset feature extraction network has converged based on the third loss value and / or the number of training times of the preset feature extraction network.

[0280] If the preset feature extraction network has not converged, the electronic device can adjust the parameters of the preset feature extraction network based on the above-mentioned third loss value, and return to execute the above-mentioned step S602, that is, continue to train the preset feature extraction network until the preset feature extraction network converges.

[0281] If the preset feature extraction network converges, the electronic device may determine the preset feature extraction network at the current moment as the trained feature extraction network.

[0282] In an optional embodiment, when determining whether the preset feature extraction network has converged, the electronic device may compare the third loss value with a preset loss value threshold. When the third loss value is greater than the preset loss value threshold, the electronic device may determine that the preset feature extraction network has not converged. When the third loss value is less than or equal to the preset loss value threshold, the electronic device may determine that the preset feature extraction network has converged.

[0283] In another optional embodiment, when determining whether the preset feature extraction network has converged, the electronic device may add 1 to the number of training times for the preset feature extraction network and compare the number of training times with a preset number threshold. When the number of training times is less than the preset number threshold, the electronic device may determine that the preset feature extraction network has not converged. When the number of training times is equal to the preset number threshold, the electronic device may determine that the preset feature extraction network has converged.

[0284] Here, the method of determining whether the preset feature extraction network has converged based on the third loss value and the number of training times of the preset feature extraction network is not described in detail.

[0285] In an optional embodiment, the electronic device may use a preset parameter adjustment algorithm to adjust the parameters of the preset feature extraction network.

[0286] The preset parameter adjustment algorithm includes, but is not limited to, a stochastic gradient descent (SGD) algorithm and an adaptive momentum estimation (ADAM) algorithm. Here, the preset parameter adjustment algorithm is not specifically limited.

[0287] In the embodiment of the present application, the convergence judgment method and parameter adjustment method of the above-mentioned preset cyclic adversarial generation network and the preset recognition network can refer to the convergence judgment method and parameter adjustment method of the above-mentioned preset feature extraction network, and no specific description is given here.

[0288] In an optional embodiment, according to the above Figure 1 The method shown in the embodiment of the present application also provides a feature extraction network training method. Figure 9 As shown, Figure 9 This is a sixth flow chart of the feature extraction network training method provided in the embodiment of the present application. The method includes the following steps.

[0289] Step S901 : Acquire a third data set, where the third data set includes a plurality of fourth sample images and a second true label of each fourth sample image.

[0290] Step S902 : For each fourth sample image, extract features of the fourth sample image using the first network model to obtain seventh feature data of the fourth sample image.

[0291] The method for extracting the seventh characteristic data may refer to the method for extracting the first characteristic data, and will not be described in detail here.

[0292] The first network model is the network model corresponding to the untrained preset feature extraction network. The first network model may be a neural network model, such as the convolutional neural network model described above. The first network model is used to extract features of objects in an input image. The first network model is not specifically limited herein.

[0293] Step S903: Input the seventh feature data of the fourth sample image into the second network model to obtain a fourth recognition result of the fourth sample image, and determine a fourth predicted label of the fourth sample image according to the fourth recognition result.

[0294] The method for determining the fourth prediction label may refer to the method for determining the first prediction label, and will not be described in detail here.

[0295] The second network model is the network model corresponding to the pre-trained preset recognition network. This second network model can be a deep learning model or a machine learning model. This second network model can be used to identify and classify objects in the image input by the first network model based on the input feature data. The second network model is not specifically limited herein.

[0296] Step S904: Calculate sixth loss values of the first network model and the second network model according to the fourth predicted labels and the second true labels corresponding to the plurality of fourth sample images.

[0297] The calculation method of the sixth loss value may refer to the calculation method of the third loss value. Here, the calculation process of the sixth loss value will not be described in detail.

[0298] Step S905: Based on the sixth loss value, adjust the parameters of the first network model and the second network model, and return to execute the step of performing feature extraction on each fourth sample image using the first network model to obtain the seventh feature data of the fourth sample image, until the first network model and the second network model converge, determine the current first network model as the preset feature extraction network, and determine the current second network model as the preset recognition network.

[0299] In an embodiment of the present application, after the electronic device calculates the sixth loss value, it can determine whether the first network model and the second network model have converged based on the sixth loss value and / or the number of training times of the first network model and the second network model.

[0300] If the first network model and the second network model do not converge, the electronic device can adjust the parameters of the first network model and the second network model based on the sixth loss value, and return to execute the above step S902, that is, re-train the first network model and the second network model until the first network model and the second network model converge.

[0301] If the first network model and the second network model converge, the electronic device may determine the first network model at the current moment as the preset feature extraction network, and determine the second network model as the preset recognition network.

[0302] The convergence judgment method and parameter adjustment method of the first network model and the second network model can refer to the convergence judgment method and parameter adjustment method of the preset feature extraction network, and will not be described in detail here.

[0303] Through the above steps S901 to S905, the electronic device can use the data in the above third data set to complete the training process of the first network model and the second network model, so that the first network model has the function of extracting features of image data, and the second network model has the ability to recognize the feature data extracted by the first network model, thereby facilitating further training of the preset feature extraction network obtained by training.

[0304] Step S906 : Acquire a first data set, where the first data set includes a plurality of first sample images and a first true label for each first sample image.

[0305] Step S907 : For each first sample image, a preset feature extraction network is used to perform feature extraction on the first sample image to obtain first feature data of the first sample image.

[0306] Step S908: input the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image to generate second feature data of the first sample image. The preset feature conversion network is trained using feature data of sample images of different image scenes.

[0307] Step S909: Adjust the parameters of the preset feature extraction network according to the recognition results of the preset recognition network for the first feature data and the second feature data corresponding to each first sample image, as well as the first true label of each first sample image, and return to the step of performing feature extraction on each first sample image using the preset feature extraction network to obtain the first feature data of the first sample image, until the preset feature extraction network converges to obtain a trained feature extraction network.

[0308] The above steps S906 to S909 are the same as the above steps S101 to S104.

[0309] Based on the same inventive concept, and in accordance with the feature extraction network training method provided in the above-mentioned embodiments of the present application, the present application also provides an image recognition method. This method can be applied to the above-mentioned electronic device as well as other electronic devices. For ease of understanding, the description will still be based on the above-mentioned electronic device as the execution subject, and this is not intended to be limiting in any way.

[0310] like Figure 10 As shown, Figure 10 A flow chart of an image recognition method provided in an embodiment of the present application is provided. The method includes the following steps.

[0311] Step S1001: Obtain a face image to be recognized.

[0312] Step S1002: input the face image to be recognized into a feature extraction network to obtain facial feature data of the face image to be recognized.

[0313] In this step, the electronic device can use the trained feature extraction network to extract features of the target face in the face image to be identified, and obtain facial feature data of the face image to be identified.

[0314] Step S1003: Obtaining a face recognition result based on the face feature data.

[0315] In this step, the electronic device uses the recognition network to determine the attribute information of the target face in the image to be recognized based on the facial feature data, and obtains a facial recognition result. For example, the electronic device can use the identity recognition network to determine the identity information of the target face in the image to be recognized.

[0316] In the above steps S1001 to S1003, since the feature extraction network is trained by feature data obtained by expanding feature level data, the feature extraction network can perform feature extraction on images to be identified in different image scenes, effectively ensuring the accuracy of the extracted feature data, thereby facilitating subsequent image processing based on the feature data, such as image recognition, image classification, etc.

[0317] Based on the same inventive concept, according to the feature extraction network training method provided in the above embodiment of the present application, the embodiment of the present application also provides a feature extraction network training device. Figure 11 As shown, Figure 11 A schematic diagram of the structure of a feature extraction network training device provided in an embodiment of the present application. The device includes the following modules.

[0318] A first acquisition module 1101 is configured to acquire a first data set, where the first data set includes a plurality of first sample images and a first true label for each first sample image;

[0319] A first extraction module 1102 is configured to perform feature extraction on each first sample image using a preset feature extraction network to obtain first feature data of the first sample image;

[0320] A first generating module 1103 is configured to input the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image to generate second feature data of the first sample image, where the preset feature conversion network is trained using feature data of sample images of different image scenes;

[0321] The first adjustment module 1104 is used to adjust the parameters of the preset feature extraction network according to the recognition results of the first feature data and the second feature data corresponding to each first sample image by the preset recognition network, as well as the first true label of each first sample image, and return to execute the step of performing feature extraction on the first sample image using the preset feature extraction network to obtain the first feature data of the first sample image for each first sample image, until the preset feature extraction network converges to obtain a trained feature extraction network.

[0322] Optionally, the preset feature conversion network is the first generator or the second generator in a pre-trained cyclic adversarial generative network;

[0323] The feature extraction network training device may further include:

[0324] a second acquisition module, configured to acquire a second data set before inputting the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image to generate the second feature data of the first sample image, the second data set including the third feature data of the second sample image of the first image scene and the fourth feature data of the third sample image of the second image scene;

[0325] A second generating module is configured to input the third feature data of the second sample image into the first generator in the preset cyclic adversarial generation network to obtain fifth feature data of the second sample image corresponding to the second image scene;

[0326] a third generating module, configured to input the fifth feature data corresponding to the second sample image into a second generator in a preset cyclic adversarial generation network to obtain first reconstructed feature data of the second sample image in the first image scene;

[0327] a fourth generating module, configured to input the fourth feature data corresponding to the third sample image into the second generator, to obtain sixth feature data corresponding to the third sample image in the first image scene;

[0328] A fifth generating module, configured to input sixth feature data corresponding to the third sample image into the first generator to obtain second reconstructed feature data of the third sample image in the second image scene;

[0329] A first calculation module is used to calculate a third loss value of a preset cyclic adversarial generative network based on the third feature data, the fourth feature data, the first reconstructed feature data, and the second reconstructed feature data;

[0330] The second adjustment module is used to adjust the parameters of the preset cyclic adversarial generation network based on the third loss value, and return to execute the step of inputting the third feature data of the second sample image into the first generator in the preset cyclic adversarial generation network to obtain the fifth feature data corresponding to the second sample image in the second image scene, until the preset cyclic adversarial generation network converges to obtain a trained cyclic adversarial generation network.

[0331] Optionally, the feature extraction network training device may further include:

[0332] a third acquisition module configured to, for each first sample image, obtain, for each target object in a third data set, a plurality of captured images of the target object in a preset image scene before extracting features of the first sample image using a preset feature extraction network to obtain first feature data of the first sample image, the third data set being a collection of training data for training the preset recognition network;

[0333] a second extraction module, configured to extract features from each collected image using a preset feature extraction network to obtain seventh feature data of the collected image;

[0334] The second calculation module is used to calculate the average value of the seventh feature data of the multiple collected images and update the initial weight value corresponding to the target object in the preset recognition network.

[0335] Optionally, the first adjustment module 1104 includes:

[0336] a first adjustment submodule, configured to adjust parameters of a preset recognition network based on a first loss value between a first recognition result, a second recognition result, and a first true label corresponding to a plurality of first sample images, to obtain an adjusted preset recognition network, wherein the first recognition result is obtained by the preset recognition network through target recognition based on the first feature data of the first sample image, and the second recognition result is obtained by the preset recognition network through target recognition based on the second feature data of the first sample image;

[0337] The second adjustment submodule is used to adjust the parameters of the preset feature extraction network according to the second loss value between the third recognition results corresponding to multiple first sample images and the first true label, and the third recognition result is obtained by the adjusted preset recognition network through target recognition based on the first feature data of the first sample image.

[0338] Optionally, the first adjustment submodule may be configured to perform target recognition on each first sample image using a preset recognition network based on first feature data of the first sample image, obtain a first recognition result for the first sample image, and determine a first predicted label for the first sample image based on the first recognition result.

[0339] Using a preset recognition network, performing target recognition based on the second feature data of the first sample image to obtain a second recognition result of the first sample image, and determining a second predicted label of the first sample image based on the second recognition result;

[0340] Calculating a fourth loss value between the first predicted label and the first true label corresponding to the first sample image;

[0341] Calculating a fifth loss value between the second predicted label and the first true label corresponding to the first sample image;

[0342] Calculating a first loss value of a preset recognition network according to the fourth loss values and the fifth loss values corresponding to the plurality of first sample images;

[0343] Based on the first loss value, the initial weight value corresponding to each target object in the preset recognition network is adjusted, and the step of performing target recognition according to the first feature data of the first sample image using the preset recognition network to obtain a first recognition result of the first sample image is returned to be executed until the preset recognition network converges to obtain the adjusted preset recognition network.

[0344] Optionally, the second adjustment submodule may be configured to, for each first sample image, use the adjusted preset recognition network to perform target recognition based on the first feature data of the first sample image, obtain a third recognition result for the first sample image, and determine a third predicted label for the first sample image based on the third recognition result;

[0345] Calculating a loss value between a third predicted label and the first true label corresponding to the plurality of first sample images as a second loss value of the preset feature extraction network;

[0346] Based on the second loss value, parameters of the preset feature extraction network are adjusted.

[0347] Optionally, the feature extraction network training device may further include:

[0348] a fourth acquisition module, configured to acquire a third data set before acquiring the first data set, where the third data set includes a plurality of fourth sample images and a second true label for each fourth sample image;

[0349] a third extraction module, configured to perform feature extraction on each fourth sample image using the first network model to obtain seventh feature data of the fourth sample image;

[0350] a first recognition module, configured to input the seventh feature data of the fourth sample image into the second network model to obtain a fourth recognition result of the fourth sample image, and determine a fourth predicted label of the fourth sample image based on the fourth recognition result;

[0351] A third calculation module is used to calculate the sixth loss value of the first network model and the second network model according to the fourth predicted labels and the second true labels corresponding to the plurality of fourth sample images;

[0352] The third adjustment module is used to adjust the parameters of the first network model and the second network model based on the sixth loss value, and return to execute the step of performing feature extraction on each fourth sample image using the first network model to obtain the seventh feature data of the fourth sample image until the first network model and the second network model converge, and determine the current first network model as the preset feature extraction network, and determine the current second network model as the preset recognition network.

[0353] Based on the same inventive concept, according to the image recognition method provided in the above embodiment of the present application, the embodiment of the present application also provides an image recognition device. Figure 12 As shown, Figure 12 A schematic diagram of the structure of an image recognition device provided in an embodiment of the present application. The device includes the following modules.

[0354] The fifth acquisition module 1201 is used to acquire a face image to be recognized;

[0355] A fourth extraction module 1202 is configured to input the face image to be recognized into a feature extraction network to obtain facial feature data of the face image to be recognized; wherein the feature extraction network is a feature extraction network trained in any of the above-mentioned feature extraction network training methods;

[0356] The recognition module 1203 is used to obtain a face recognition result based on the face feature data.

[0357] Through the device provided in the embodiment of the present application, a preset feature extraction network can be used to perform feature extraction on each first sample image in the first data set to obtain first feature data of the first sample image. For each first sample image, the preset feature conversion network corresponding to the image scene of the first sample image is used to generate second feature data of the first sample image in different image scenes, thereby adjusting the parameters of the preset feature extraction network based on the recognition results of the first feature data and the second feature data corresponding to each first sample image by the preset recognition network, as well as the first true label of each first sample image, and repeating the training process of the preset feature extraction network until the preset feature extraction network converges to obtain a trained feature extraction network.

[0358] Through the preset feature conversion network corresponding to the image scene of each first sample image, the second feature data of the first sample image in different image scenes can be generated, that is, the feature data corresponding to the sample image in different image scenes are generated in the network training stage, and the feature data expansion of the feature data of the sample image at the feature level is realized, which effectively improves the applicability of the feature extraction network trained based on the feature data of multiple different image scenes to images of different image scenes, so that the trained feature extraction network can extract features from images collected in different image scenes, thereby improving the performance of the trained feature extraction network.

[0359] In addition, after using the preset feature conversion network to perform feature conversion on the feature data of each sample image to obtain the second feature data, the parameter adjustment process of the preset feature extraction network is affected not only by the first feature data corresponding to each sample image, but also by the second feature data corresponding to each sample image. This makes the expanded second feature data closer to the feature data corresponding to the real image scene, thereby improving the accuracy of the feature extraction network trained based on feature data of multiple different image scenes.

[0360] The present application also provides an electronic device, such as Figure 13 As shown, it includes a processor 1301, a communication interface 1302, a memory 1303 and a communication bus 1304, wherein the processor 1301, the communication interface 1302, and the memory 1303 communicate with each other through the communication bus 1304.

[0361] Memory 1303, used for storing computer programs;

[0362] The processor 1301 is configured to execute the program stored in the memory 1303 to implement the following steps:

[0363] Acquire a first data set, where the first data set includes a plurality of first sample images and a first true label for each first sample image;

[0364] For each first sample image, using a preset feature extraction network, perform feature extraction on the first sample image to obtain first feature data of the first sample image;

[0365] Inputting the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image to generate second feature data of the first sample image, where the preset feature conversion network is trained using feature data of sample images of different image scenes;

[0366] According to the recognition result of the preset recognition network for the first feature data and the second feature data corresponding to each first sample image, as well as the first true label of each first sample image, the parameters of the preset feature extraction network are adjusted, and the step of performing feature extraction on each first sample image using the preset feature extraction network to obtain the first feature data of the first sample image is returned to execution until the preset feature extraction network converges to obtain a trained feature extraction network.

[0367] The present application also provides an electronic device, such as Figure 14 As shown, it includes a processor 1401, a communication interface 1402, a memory 1403 and a communication bus 1404, wherein the processor 1401, the communication interface 1402, and the memory 1403 communicate with each other through the communication bus 1404.

[0368] Memory 1403, used for storing computer programs;

[0369] The processor 1401 is configured to execute the program stored in the memory 1403 to implement the following steps:

[0370] Obtain the face image to be recognized;

[0371] Inputting the face image to be recognized into a feature extraction network to obtain facial feature data of the face image to be recognized; wherein the feature extraction network is a feature extraction network trained based on any of the above-mentioned feature extraction network training methods;

[0372] Based on the facial feature data, the face recognition result is obtained.

[0373] The electronic device provided by the embodiment of the present application can use a preset feature extraction network to perform feature extraction on each first sample image in the first data set to obtain first feature data of the first sample image. For each first sample image, the preset feature conversion network corresponding to the image scene of the first sample image is used to generate second feature data of the first sample image in different image scenes, thereby adjusting the parameters of the preset feature extraction network based on the recognition results of the first feature data and the second feature data corresponding to each first sample image of the preset recognition network, as well as the first true label of each first sample image, and repeating the training process of the preset feature extraction network until the preset feature extraction network converges to obtain a trained feature extraction network.

[0374] Through the preset feature conversion network corresponding to the image scene of each first sample image, the second feature data of the first sample image in different image scenes can be generated, that is, the feature data corresponding to the sample image in different image scenes are generated in the network training stage, and the feature data expansion of the feature data of the sample image at the feature level is realized, which effectively improves the applicability of the feature extraction network trained based on the feature data of multiple different image scenes to images of different image scenes, so that the trained feature extraction network can extract features from images collected in different image scenes, thereby improving the performance of the trained feature extraction network.

[0375] In addition, after using the preset feature conversion network to perform feature conversion on the feature data of each sample image to obtain the second feature data, the parameter adjustment process of the preset feature extraction network is affected not only by the first feature data corresponding to each sample image, but also by the second feature data corresponding to each sample image. This makes the expanded second feature data closer to the feature data corresponding to the real image scene, thereby improving the accuracy of the feature extraction network trained based on feature data of multiple different image scenes.

[0376] The communication bus mentioned in the electronic device mentioned above may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0377] The communication interface is used for communication between the above electronic device and other devices.

[0378] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk memory. Alternatively, the memory may be at least one storage device located away from the processor.

[0379] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0380] Based on the same inventive concept, according to the feature extraction network training method provided in the above-mentioned embodiments of the present application, the embodiments of the present application provide a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of any of the above-mentioned feature extraction network training methods.

[0381] Based on the same inventive concept, according to the feature extraction network training method provided in the above-mentioned embodiments of the present application, the embodiments of the present application provide a computer program product comprising instructions, which, when run on a computer, enables the computer to execute any one of the feature extraction network training methods in the above-mentioned embodiments.

[0382] Based on the same inventive concept, according to the image recognition method provided in the above-mentioned embodiments of the present application, the embodiments of the present application also provide a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned image recognition methods are implemented.

[0383] Based on the same inventive concept, according to the image recognition method provided in the above-mentioned embodiments of the present application, the embodiments of the present application also provide a computer program product containing instructions, which, when run on a computer, enables the computer to execute any image recognition method in the above-mentioned embodiments.

[0384] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0385] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0386] Each embodiment in this specification is described in a related manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, for embodiments such as apparatus, electronic devices, computer-readable storage media, and computer program products, since they are generally similar to the method embodiments, their descriptions are relatively simplified. For relevant portions, reference can be made to the descriptions of the method embodiments.

[0387] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A feature extraction network training method, characterized in that: The method comprises: Acquire a first data set, where the first data set includes a plurality of first sample images and a first true label of an object in each first sample image; For each first sample image, using a preset feature extraction network, extract features of the target object in the first sample image to obtain first feature data of the first sample image; Inputting the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image to generate second feature data of the first sample image, wherein the preset feature conversion network is trained using feature data of sample images of different image scenes; Adjusting parameters of the preset feature extraction network according to a recognition result of the preset recognition network for the first feature data and the second feature data corresponding to each first sample image, and the first true label of each first sample image, and returning to the step of extracting features of each first sample image using the preset feature extraction network to obtain the first feature data of the first sample image, until the preset feature extraction network converges to obtain a trained feature extraction network; The preset feature conversion network is the first generator or the second generator in a pre-trained cyclic adversarial generation network; Before inputting the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image to generate the second feature data of the first sample image, the method further includes: Acquire a second data set, the second data set including third feature data of a second sample image of the first image scene and fourth feature data of a third sample image of the second image scene; Inputting the third feature data of the second sample image into the first generator in the preset recurrent adversarial generative network to obtain fifth feature data corresponding to the second sample image in the second image scene; Inputting the fifth feature data corresponding to the second sample image into the second generator in the preset cyclic adversarial generative network to obtain first reconstructed feature data of the second sample image in the first image scene; Inputting the fourth feature data corresponding to the third sample image into the second generator to obtain the sixth feature data corresponding to the third sample image in the first image scene; inputting the sixth feature data corresponding to the third sample image into the first generator to obtain second reconstructed feature data of the third sample image in the second image scene; Calculating a first loss value of the preset cyclic adversarial generative network based on the third feature data, the fourth feature data, the first reconstructed feature data, and the second reconstructed feature data; Adjust the parameters of the preset cyclic adversarial generative network based on the first loss value, and return to the step of inputting the third feature data of the second sample image into the first generator in the preset cyclic adversarial generative network to obtain the fifth feature data corresponding to the second sample image in the second image scene, until the preset cyclic adversarial generative network converges to obtain a trained cyclic adversarial generative network.

2. The method according to claim 1, characterized in that Before extracting features of the target object in each first sample image using a preset feature extraction network to obtain first feature data of the first sample image, the method further includes: For each target object in a third data set, obtaining a plurality of captured images of the target object in a preset image scene, wherein the third data set is a set of training data for training the preset recognition network; For each collected image, extract features of the collected image using the preset feature extraction network to obtain seventh feature data of the collected image; An average value of the seventh feature data of the plurality of collected images is calculated, and an initial weight value corresponding to the target object in the preset recognition network is updated.

3. The method according to claim 1, characterized in that The step of adjusting the parameters of the preset feature extraction network according to the recognition result of the preset recognition network for the first feature data and the second feature data corresponding to each first sample image, and the first true label of each first sample image, includes: Adjusting parameters of a preset recognition network according to a second loss value between the first recognition results, the second recognition results, and the first true label corresponding to the plurality of first sample images to obtain an adjusted preset recognition network, wherein the first recognition result is obtained by the preset recognition network through target recognition based on the first feature data of the first sample image, and the second recognition result is obtained by the preset recognition network through target recognition based on the second feature data of the first sample image; According to the third loss value between the third recognition results corresponding to the multiple first sample images and the first true label, the parameters of the preset feature extraction network are adjusted, and the third recognition result is obtained by the adjusted preset recognition network through target recognition based on the first feature data of the first sample image.

4. The method according to claim 2 or 3, characterized in that The step of adjusting the parameters of the preset recognition network according to the first recognition results, the second recognition results, and the second loss value between the first true label corresponding to the plurality of first sample images to obtain the adjusted preset recognition network includes: For each first sample image, using a preset recognition network, perform target recognition based on first feature data of the first sample image to obtain a first recognition result of the first sample image, and determine a first predicted label of the first sample image based on the first recognition result; Using the preset recognition network, performing target recognition based on the second feature data of the first sample image to obtain a second recognition result of the first sample image, and determining a second predicted label of the first sample image based on the second recognition result; Calculating a fourth loss value between the first predicted label and the first true label corresponding to the first sample image; Calculating a fifth loss value between the second predicted label and the first true label corresponding to the first sample image; Calculating a second loss value of the preset recognition network according to the fourth loss values and the fifth loss values corresponding to the plurality of first sample images; Based on the second loss value, the initial weight value corresponding to each target object in the preset recognition network is adjusted, and the step of performing target recognition according to the first feature data of the first sample image using the preset recognition network to obtain a first recognition result of the first sample image is returned to execution for each first sample image, until the preset recognition network converges to obtain an adjusted preset recognition network.

5. The method according to claim 4, characterized in that The step of adjusting the parameters of the preset feature extraction network according to the third loss value between the third recognition results corresponding to the plurality of first sample images and the first true label includes: For each first sample image, using the adjusted preset recognition network, perform target recognition based on the first feature data of the first sample image to obtain a third recognition result for the first sample image, and determine a third predicted label for the first sample image based on the third recognition result; Calculating a loss value between a third predicted label and a first true label corresponding to the plurality of first sample images as a third loss value of the preset feature extraction network; Based on the third loss value, parameters of the preset feature extraction network are adjusted.

6. The method according to claim 1, characterized in that Before acquiring the first data set, the method further includes: Acquire a third data set, where the third data set includes a plurality of fourth sample images and a second true label for each fourth sample image; For each fourth sample image, extract features of the fourth sample image using the first network model to obtain seventh feature data of the fourth sample image; inputting the seventh feature data of the fourth sample image into the second network model to obtain a fourth recognition result of the fourth sample image, and determining a fourth predicted label of the fourth sample image based on the fourth recognition result; Calculating a sixth loss value of the first network model and the second network model according to the fourth predicted labels and the second true labels corresponding to the plurality of fourth sample images; Based on the sixth loss value, the parameters of the first network model and the second network model are adjusted, and the step of performing feature extraction on each fourth sample image using the first network model to obtain the seventh feature data of the fourth sample image is returned to execution until the first network model and the second network model converge, and the current first network model is determined as the preset feature extraction network, and the current second network model is determined as the preset recognition network.

7. An image recognition method, characterized in that: The method comprises: Obtain the face image to be recognized; Inputting the face image to be identified into a feature extraction network to obtain facial feature data of the face image to be identified; wherein the feature extraction network is the feature extraction network trained in any one of the methods of claims 1 to 6; Based on the facial feature data, a face recognition result is obtained.

8. A feature extraction network training device, characterized in that: The device comprises: A first acquisition module is configured to acquire a first data set, wherein the first data set includes a plurality of first sample images and a first true label of an object in each first sample image; a first extraction module, configured to extract features of the target object in each first sample image using a preset feature extraction network to obtain first feature data of the first sample image; a first generating module, configured to input the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image, to generate second feature data of the first sample image, wherein the preset feature conversion network is trained using feature data of sample images of different image scenes; a first adjustment module, configured to adjust parameters of the preset feature extraction network based on a recognition result of the first feature data and the second feature data corresponding to each first sample image by the preset recognition network, and the first true label of each first sample image, and return to executing the step of extracting features of each first sample image using the preset feature extraction network to obtain the first feature data of the first sample image, until the preset feature extraction network converges to obtain a trained feature extraction network; The preset feature conversion network is the first generator or the second generator in a pre-trained cyclic adversarial generation network; Before inputting the first feature data of the first sample image into a preset feature conversion network corresponding to the image scene of the first sample image to generate the second feature data of the first sample image, the apparatus further includes: a second acquisition module, configured to acquire a second data set, wherein the second data set includes third feature data of the second sample image of the first image scene and fourth feature data of the third sample image of the second image scene; a second generating module, configured to input the third feature data of the second sample image into the first generator in the preset cyclic adversarial generation network to obtain fifth feature data corresponding to the second sample image in the second image scene; a third generating module, configured to input the fifth feature data corresponding to the second sample image into the second generator in the preset cyclic adversarial generative network to obtain first reconstructed feature data of the second sample image in the first image scene; a fourth generating module, configured to input the fourth feature data corresponding to the third sample image into the second generator, to obtain sixth feature data corresponding to the third sample image in the first image scene; a fifth generating module, configured to input sixth feature data corresponding to the third sample image into the first generator to obtain second reconstructed feature data of the third sample image in the second image scene; A first calculation module is configured to calculate a first loss value of the preset cyclic adversarial generative network based on the third feature data, the fourth feature data, the first reconstructed feature data, and the second reconstructed feature data; A second adjustment module is used to adjust the parameters of the preset cyclic adversarial generative network based on the first loss value, and return to execute the step of inputting the third feature data of the second sample image into the first generator in the preset cyclic adversarial generative network to obtain the fifth feature data corresponding to the second sample image in the second image scene, until the preset cyclic adversarial generative network converges to obtain a trained cyclic adversarial generative network.

9. An image recognition device, characterized in that: The device comprises: A fifth acquisition module is used to acquire a face image to be recognized; a fourth extraction module, configured to input the face image to be identified into a feature extraction network to obtain facial feature data of the face image to be identified; wherein the feature extraction network is a feature extraction network trained in any one of claims 1 to 6; The recognition module is used to obtain a face recognition result based on the face feature data.

Images