Feature extraction model training method, object recognition method, device and equipment

Abstract

The embodiments of the present application provide a method for training a feature extraction model, an object recognition method, an apparatus and equipment, and relate to the fields of artificial intelligence, big data, multimedia and cloud technology. The training method comprises: obtaining a first training set and n first neural network models to be trained, each of which includes a first initial feature extraction module, the first neural network model including a first initial feature extraction module; using the first training set, training each first initial feature extraction module separately to obtain n first object feature extraction modules; obtaining a second training set and a second neural network model to be trained, including a second initial feature extraction module, and training the second neural network model based on the trained n first neural network models and the second training set to obtain a trained object feature extraction model. Based on the method provided in the embodiments of the present application, the performance of the trained object feature extraction model can be effectively improved.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence, cloud technology, and image recognition technology. Specifically, the present application relates to a training method for a feature extraction model, an object recognition method, corresponding devices, and electronic equipment. Background Art

[0002] With the advancement of AI research and technology, it has been studied and applied in many fields. Machine learning is the core of AI and the fundamental way to make computers intelligent. Through machine learning, neural network models can be trained to meet application needs.

[0003] The target recognition model is a neural network model that is currently needed in many fields. This model can identify objects contained in images. Its principle is to extract features of the image to be identified through the object feature extraction module in the model, and predict the recognition result of the object based on the features of the object contained in the image (that is, the feature vector representing the object).

[0004] The performance of an object recognition model is crucially influenced by the expressive power of the object features extracted by its object feature extraction module. Currently, various training methods exist for training the object feature extraction module of object recognition models. While most methods can produce object recognition models that meet general requirements, the accuracy of the recognition results still needs to be improved, especially for images of suboptimal quality. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a method for training a feature extraction model that can effectively improve the performance of the object feature extraction model, as well as a corresponding object recognition method, a corresponding device, and an electronic device. In order to achieve the above purpose, the technical solutions provided by the embodiments of the present application are as follows:

[0006] In one aspect, an embodiment of the present application provides a method for training a feature extraction model, the method comprising:

[0007] Obtaining a first training set and n first neural network models to be trained, where n is greater than or equal to 2, wherein the first training set includes a plurality of first sample images with labels, wherein the labels represent actual recognition results of sample objects contained in the first sample images, and the first neural network model includes a first initial feature extraction module;

[0008] Using the first training set, training each first initial feature extraction module separately to obtain n trained first object feature extraction modules;

[0009] Obtaining a second training set and a second neural network model to be trained, wherein the second training set includes a plurality of second sample images containing sample objects, and the second neural network model includes a second initial feature extraction module;

[0010] Repeating the first training operation on the second neural network model until the first training end condition is met to obtain a trained object feature extraction model, wherein the first training operation includes:

[0011] For each of the second sample images, the image is input into the n first object feature extraction modules to obtain n first object feature vectors of the image, and the image is input into the second initial feature extraction module to obtain the second object feature vector of the image. Based on the difference between the n first object feature vectors and the second object feature vector of the image, the first training loss corresponding to the image is determined; based on the first training loss corresponding to each of the second sample images, the first total training loss corresponding to the second neural network model is determined. If the first training end condition is not met, the model parameters of the second neural network model are adjusted based on the first total training loss.

[0012] In another aspect, an embodiment of the present application provides an object recognition method, the method comprising:

[0013] Acquire an image to be identified, wherein the image to be identified includes an object to be identified;

[0014] Extracting features from the image to be identified by calling an object feature extraction model to obtain an object feature vector of the object to be identified;

[0015] Determining a recognition result of the object to be recognized based on the object feature vector of the object to be recognized;

[0016] The object feature extraction model is obtained by training using any training method provided in any optional embodiment of the present application.

[0017] On the other hand, an embodiment of the present application provides a training device for an object feature extraction model, the device comprising:

[0018] a training data acquisition module, configured to acquire a first training set and n first neural network models to be trained, where n ≥ 2, wherein the first training set includes a plurality of first sample images with labels, wherein the labels represent actual recognition results of sample objects contained in the first sample images, and the first neural network model includes a first initial feature extraction module; and a second training set and a second neural network model to be trained, wherein the second training set includes a plurality of second sample images containing sample objects, and the second neural network model includes a second initial feature extraction module;

[0019] a first model training module, configured to train each first initial feature extraction module using the first training set to obtain n trained first object feature extraction modules;

[0020] The second model training module is configured to repeatedly perform a first training operation on the second neural network model until a first training end condition is satisfied, thereby obtaining a trained object feature extraction model. The first training operation includes:

[0021] For each of the second sample images, the image is input into the n first object feature extraction modules to obtain n first object feature vectors of the image, and the image is input into the second initial feature extraction module to obtain the second object feature vector of the image. Based on the difference between the n first object feature vectors and the second object feature vector of the image, the first training loss corresponding to the image is determined; based on the first training loss corresponding to each of the second sample images, the first total training loss corresponding to the second neural network model is determined. If the first training end condition is not met, the model parameters of the second neural network model are adjusted based on the first total training loss.

[0022] Optionally, for each second sample image, the second model training module may be configured to:

[0023] The n first object feature vectors corresponding to the image are fused to obtain a first fused feature vector; and a first training loss corresponding to the image is determined based on a difference between the first fused feature vector and the second object feature vector corresponding to the image.

[0024] Optionally, for each second sample image, the n first object feature vectors corresponding to the image are obtained in the following manner:

[0025] For each of the first object feature extraction modules, extract features from the image using the module to obtain a first feature map, and perform feature mapping on the first feature map to obtain a first object feature vector corresponding to the module;

[0026] For each of the second sample images, the second model training module may be configured to:

[0027] For each first object feature vector of the image, determine a first confidence of the feature vector based on a first feature map corresponding to the feature vector; for each first object feature vector corresponding to the image, weight the feature vector using the first confidence of the feature vector to obtain a weighted first object feature vector; fuse the n weighted first object feature vectors corresponding to the image to obtain a second fused feature vector; and determine a first training loss corresponding to the image based on a difference between the second fused feature vector and the second object feature vector corresponding to the image.

[0028] Optionally, each of the first neural network models further includes a first uncertainty estimation module to be trained corresponding to the first initial feature extraction module. After obtaining the trained n first object feature extraction modules, the first model training module may further be used to perform the following operations:

[0029] Obtaining a third training set, the third training set comprising a plurality of sample subsets and a class feature vector of each sample subset, each sample subset comprising at least one third sample image of a sample object, the class feature vector representing a true feature vector of the sample object corresponding to the corresponding sample subset;

[0030] For each first uncertainty estimation module to be trained, repeatedly performing a second training operation on the uncertainty estimation module based on the third training set until a second training end condition is met, thereby obtaining a trained first uncertainty estimation module, wherein the second training operation includes:

[0031] For each of the third sample images, feature extraction is performed on the image through the first object feature extraction module corresponding to the uncertainty estimation module to obtain a third feature map of the image, and a third object feature vector of the image is obtained based on the feature map. The third feature map is input into the uncertainty estimation module to obtain a third confidence of the third object feature vector. The second training loss corresponding to the image is determined based on the third object feature vector, the third confidence, and the category feature vector corresponding to the image; based on the second training loss corresponding to each of the third sample images, the second total training loss corresponding to the uncertainty estimation model is determined. If the second training end condition is not met, the model parameters of the uncertainty estimation module are adjusted based on the second total training loss; wherein, the first confidence of each of the first object feature vectors is obtained by the corresponding trained first uncertainty estimation module.

[0032] Optionally, for each second sample image, the second object feature vector of the image is obtained in the following manner:

[0033] Performing feature extraction on the image by the second initial feature extraction module to obtain a second feature map, and performing feature mapping on the second feature map to obtain a second object feature vector of the image;

[0034] The second neural network model further includes a second uncertainty estimation module to be trained. For each second sample image, the first training operation further includes:

[0035] Inputting the second feature map corresponding to the image into the second uncertainty estimation module to be trained to obtain a second confidence level corresponding to the second object feature vector of the image;

[0036] For each second sample image, the second model training module may be configured to:

[0037] Determine a fusion confidence of a second fusion feature vector corresponding to the image; and determine a first training loss corresponding to the image based on a difference between the second fusion feature vector corresponding to the image and the second object feature vector, and a difference between the fusion confidence corresponding to the image and the second confidence.

[0038] Optionally, for each second sample image, the second model training module may determine the fusion confidence of the second fused feature vector corresponding to the image by any one of the following:

[0039] Determining a fusion confidence of the second fused feature vector based on the second fused feature vector corresponding to the image;

[0040] The n first confidences corresponding to the image are fused to determine a fusion confidence of a second fused feature vector of the image.

[0041] Optionally, for each second sample image, when the second model training module fuses the weighted n first object feature vectors corresponding to the image to obtain a second fused feature vector, the second fused feature vector can be obtained by the following expression:

[0042]

[0043] in, represents the second fusion feature vector, n represents the number of the first neural network model, μ i Indicates that the second sample image corresponds to the first object feature vector of the i-th first neural network model, 1≤i≤n, k i represents μ i The first confidence level of , || ||2 is the L2 norm calculation symbol.

[0044] Optionally, the fusion confidence corresponding to each second sample image is determined by the following expression:

[0045]

[0046] in, represents the fusion confidence, n represents the number of the first neural network model, μ i Indicates that the second sample image corresponds to the first object feature vector of the i-th first neural network model, 1≤i≤n, k i represents μ i The first confidence level of , ||||2 is the L2 norm calculation symbol.

[0047] Optionally, the first training loss corresponding to each second sample image is determined by the following expression:

[0048]

[0049] in, Represents the second fused feature vector corresponding to the image, express The fusion confidence, μ φ (x) represents the second object feature vector of the image, represents μ φ The second confidence level of (x).

[0050] Optionally, for each of the sample subsets, the category feature vector of the sample subset includes the category feature vector of each first neural network model corresponding to the sample subset;

[0051] For each first uncertainty estimation module, when the first model training module determines the second training loss corresponding to each second sample image based on the third object feature vector, the third confidence level, and the category feature vector corresponding to the image, the first model training module may be used to:

[0052] Determine a second training loss corresponding to the image based on a third object feature vector of the image, a third confidence level, and a category feature vector of the image corresponding to the first neural network model to which the uncertainty estimation module belongs;

[0053] The category feature vector of each sample subset can be obtained by any of the following methods:

[0054] For each of the first neural network models, inputting each third sample image in the sample subset into the first object feature extraction module of the model to obtain an object feature vector for each third sample image; fusing the object feature vectors of the third sample images in the sample subset to obtain a category feature vector of the sample subset corresponding to the first neural network model;

[0055] For each of the first neural network models, the model parameters corresponding to the category of the sample subset in the model parameters of the first object feature extraction module of the model are used as the category feature vector of the sample subset corresponding to the first neural network model, wherein each of the sample subsets corresponds to a category.

[0056] On the other hand, an embodiment of the present application provides an object recognition device, the device comprising:

[0057] The module for acquiring an image to be processed is used to acquire an image to be identified, wherein the image to be identified includes an object to be identified;

[0058] An image recognition module is configured to extract features from the image to be recognized by calling an object feature extraction model to obtain an object feature vector of the object to be recognized, and determine a recognition result of the object to be recognized based on the object feature vector of the object to be recognized;

[0059] The object feature extraction model is obtained by training using any training method provided in any optional embodiment of the present application.

[0060] Optionally, the image recognition module can be used to:

[0061] The object feature extraction module performs feature extraction on the image to be identified to obtain a feature map of the image to be identified, and based on the feature map, obtains an object feature vector of the object to be identified and a confidence level of the object feature vector; based on the object feature vector of the object to be identified and the confidence level of the object feature vector, determines an identification result of the object to be identified.

[0062] Optionally, the module for acquiring images to be processed may also be used for:

[0063] Acquire an image to be compared, wherein the image to be compared includes an object to be compared; obtain an object feature vector of the object to be compared and a confidence level of the object feature vector using the object feature extraction model;

[0064] The image recognition module can be used to determine whether the object to be identified and the object to be identified are the same object based on the object feature vector of the object to be compared and the confidence of the object feature vector, and the object feature vector of the object to be identified and the confidence of the object feature vector.

[0065] On the other hand, an embodiment of the present application further provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the method provided in any optional embodiment of the present application by executing the computer program.

[0066] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method provided by any optional embodiment of the present application is implemented.

[0067] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the method provided by any optional embodiment of the present application.

[0068] The beneficial effects of the technical solution provided by the embodiments of the present application are as follows:

[0069] The training method provided in the embodiment of the present application can first obtain multiple first object feature extraction modules based on the first training set with labels. On this basis, the multiple first object feature extraction modules can be used as guidance models to guide the training of the second neural network model to be trained based on the second training set, thereby obtaining a high-performance object feature extraction model. Since the multiple first object feature extraction modules have been trained based on the training set, when the multiple first object feature extraction modules are used as guidance models to guide the training of the second neural network model, the finally trained feature extraction model can integrate the model capabilities of the multiple first object feature extraction modules, thereby effectively improving the performance of the finally obtained object feature extraction model, and better meeting the actual application requirements. When the trained object feature extraction model is used for image feature extraction, the expression ability of the extracted features can be effectively improved, thereby effectively improving the accuracy of the recognition results when performing object recognition based on the extracted features. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application.

[0071] Figure 1 A flowchart of a method for training a feature extraction model provided in an embodiment of the present application;

[0072] Figure 2 A schematic diagram of a flow chart of a training method for a second neural network model provided in an embodiment of the present application;

[0073] Figure 3 A flowchart of an object recognition method provided in an embodiment of the present application;

[0074] Figure 4 A schematic diagram of the structure of an object recognition system provided in an embodiment of the present application;

[0075] Figure 5 A diagram illustrating the implementation principle of training and deploying a neural network model is provided in an embodiment of the present application;

[0076] Figure 6 Based on the embodiment of this application Figure 4 A schematic diagram of an implementation flow of an object recognition method of the system shown;

[0077] Figure 7a and Figure 7b A schematic diagram of the principle of a training method for a first neural network model provided in an embodiment of the present application;

[0078] Figure 8 A schematic diagram of integrating multiple features provided in an embodiment of the present application;

[0079] Figure 9 A schematic diagram of the training principle of a second neural network model provided in an embodiment of the present application;

[0080] Figure 10 A schematic diagram of the implementation principle of the deployment and application stage of an object recognition model provided in an embodiment of the present application;

[0081] Figure 11 This is a schematic structural diagram of a training device provided in an embodiment of the present application;

[0082] Figure 12 A schematic structural diagram of an object recognition device provided in an embodiment of the present application;

[0083] Figure 13 A schematic structural diagram of an electronic device applicable to an embodiment of the present application. DETAILED DESCRIPTION

[0084] The following describes the embodiments of the present application in conjunction with the accompanying drawings. It should be understood that the embodiments described below in conjunction with the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0085] Those skilled in the art will understand that, unless otherwise stated, the singular forms "a", "an", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements, and / or components, but do not exclude implementation as other features, information, data, steps, operations, elements, components, and / or combinations thereof supported by the present technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the element can be directly connected or coupled to the other element, or it can refer to the element and the other element establishing a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used here can include wireless connection or wireless coupling. The term "and / or" used here indicates at least one of the items defined by the term, for example, "A and / or B" can be implemented as "A", or as "B", or as "A and B". When describing multiple (two or more) items, if the relationship between the multiple items is not clearly defined, the multiple items may refer to one, multiple or all of the multiple items. For example, the description of "parameter A includes A1, A2, A3" can be implemented as parameter A including A1 or A2 or A3, and can also be implemented as parameter A including at least two of the three items A1, A2, and A3.

[0086] In the field of artificial intelligence, in order to obtain a neural network model that meets application requirements, the model needs to be trained. The target recognition model (object recognition model) is a neural network model that is currently needed in many fields. The model can identify objects contained in an image (such as people, animals, or specific parts of people, such as faces). Its principle is to extract features from the image to be identified through the object feature extraction module in the model, extract the features of the object contained in the image (that is, the feature vector that represents the object), and then predict the recognition result of the object based on the features of the extracted object. For example, the features of the object to be identified and the features of the reference object can be calculated for similarity, and the calculation results can be used to determine whether the object to be identified and the reference object are the same object. The performance of the target recognition model depends largely on the performance of the object feature extraction module in the object recognition model. The better the expression ability of the features of the object extracted by the object feature extraction module, the more accurate the recognition result.

[0087] In current technologies, various training methods exist for object recognition models. Most of these methods produce models that have good expressive power for features extracted from high-quality (e.g., frontal, high-definition) object images (e.g., facial images). However, for images with blur or various types of noise (e.g., occlusion, large poses), the expressive power of features extracted by models trained using these methods still needs to be improved, resulting in inaccurate recognition results. In actual applications, some human images fed into the model may contain blur or various uncontrollable noise, which reduces the model's recognition accuracy.

[0088] This application provides a training method for an object feature extraction model in order to improve the recognition accuracy of the model. The performance of the feature extraction model trained by the training method provided in the embodiment of this application is significantly improved compared with the performance of the model trained by the prior art. The feature extraction model trained by the training method provided in the embodiment of this application is used as the feature extraction model of the target recognition model for image recognition, which can effectively improve the recognition accuracy.

[0089] The solution provided in the embodiment of the present application relates to artificial intelligence (AI) technology, specifically to the training of neural network models and the application of neural network models. For example, based on the training method provided in the embodiment of the present application, the initial neural network model can be trained using machine learning (ML) to obtain a feature extraction model with better performance. After training a feature extraction model that meets the application requirements, feature extraction can be performed on the image to be identified based on the model, and the recognition result of the object can be obtained based on the feature vector of the object extracted.

[0090] Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is the study of the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making. With the research and advancement of AI technology, it has been researched and applied in a variety of fields, including common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, autonomous driving, drones, robotics, smart healthcare, intelligent customer service, connected vehicles, autonomous driving, and smart transportation. We believe that as the technology develops, this technology will be applied in even more fields and play an increasingly important role.

[0091] Optionally, the data processing involved in the methods provided in the embodiments of the present application can be implemented based on cloud technology. For example, the training method provided in the present application can be implemented based on cloud technology, and the various data calculations involved in the training process can be performed using cloud computing. Optionally, the storage of the various training sets used in the training process can also be performed using cloud storage.

[0092] The object feature extraction model training method or object recognition method provided in the embodiment of the present application can be executed by any electronic device, such as a terminal device or a server. For example, the trained object feature extraction model can be used as a feature extraction module of the target recognition model and deployed to the server. The user can collect an image of the object to be identified through the terminal device and send the image to the server. The server can perform feature extraction on the image by calling the target recognition model and obtain the recognition result of the object in the image based on the extracted feature vector. For example, the database of the server can store multiple images of objects to be compared or feature vectors of each reference object obtained by performing feature extraction on these images. After the server performs feature extraction on the image to be identified, it can determine which reference object the object in the image to be identified is based on the similarity between the extracted feature vector and the feature vectors of each reference object stored in the database.

[0093] Among them, the above-mentioned server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The above-mentioned user terminal (also referred to as user equipment) can be a smart phone, a tablet computer, a laptop computer, a desktop computer, an intelligent voice interaction device (such as a smart speaker), a wearable electronic device (such as a smart watch), a car terminal, a smart home appliance (such as a smart TV), an AR / VR device, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected through wired or wireless communication, and this application does not limit this.

[0094] Optionally, the object recognition method provided in the embodiment of the present application can be implemented as an independent application or a functional module / plug-in of an application. For example, the application can be a recognition program in a special object recognition system (such as a face recognition system), or other applications with object recognition functions. Through this application, the recognition class of the object to be recognized can be realized.

[0095] It should be noted that, in actual applications, the embodiment of this application does not limit the type of object, which may include but is not limited to people, faces or other objects. Accordingly, the sample images and images to be identified in the embodiment of this application are images containing objects.

[0096] In the optional embodiments of the present application, the data related to the object (such as the user) involved, when the embodiments of the present application are applied to specific products or technologies, need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, this data must be obtained with the authorization and consent of the object and in compliance with the relevant laws, regulations and standards of the country and region.

[0097] The following first introduces and explains some relevant technologies or technical terms involved in the embodiments of this application.

[0098] Hyperspherical space: A feature mapping space that maps the feature space of an object image (such as a face image) to a hyperspherical space can achieve relatively good performance. This indicates that among all basic geometric spaces, the hyperspherical space is more suitable for the feature representation of object images.

[0099] Gallery: Also known as a reference image set or prototype image set, it refers to a dataset containing image subsets of multiple objects, where each object's image subset contains one or more images of the object.

[0100] Test set (query): Also known as the query image set, the gallery set and the test set (query) can be used to test the model. For images in the test set, the similarity between the feature vectors of the query image and the gallery image can be calculated to match them and determine which gallery image the object in the query image corresponds to.

[0101] Model ensemble: Model ensemble is the process of fusing multiple trained models to achieve multi-model fusion in a certain way, so that the final result can "complement each other's strengths and weaknesses", integrate the learning capabilities of each model, and improve the generalization ability of the final model.

[0102] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on, or combine with each other, and the same terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0103] Figure 1A flow chart of a method for training an object feature extraction model provided in an embodiment of the present application is shown. The method can be executed by any electronic device, such as a server, such as a cloud server. The following is an example of a training server as the execution subject. Figure 1 As shown, the method includes the following steps S110 to S140.

[0104] Step S110: Obtain a first training set and n first neural network models to be trained, where n≥2, and the first neural network model includes a first initial feature extraction module.

[0105] The first training set includes multiple labeled first sample images, and the label of each first sample image represents the actual recognition result of the sample object contained in the first sample image. In other words, the first sample image is an image containing the sample object, and the image label represents the identification of the sample object. The first training set is a training data set for each first neural network model, which includes multiple training samples, namely the above-mentioned first sample images. The embodiment of this application does not limit the method for obtaining the first training data set.

[0106] In the embodiment of the present application, the first initial feature extraction module in the first neural network model is a feature extraction module to be trained, which is used to extract features from the image. The specific model structure of the first initial feature extraction module is not limited in the embodiment of the present application. Optionally, the first initial feature extraction module can be a deep feature extraction module, that is, a deep neural network for feature extraction.

[0107] Optionally, the first initial feature extraction module may include a cascaded feature map extraction module and a feature mapping module, the feature map extraction module is used to extract the feature map of the input image, and the feature mapping module is used to map the feature map to a specified feature space to obtain a feature vector of the object in the image. In some embodiments described below, the feature map extraction module of the first initial feature extraction module may also be referred to as a deep network unit module. The structure of the feature map extraction module may generally be a convolutional neural network (CNN), which may include operations such as convolution calculation, nonlinear activation function (Relu) calculation, and pooling calculation. The function of this module is to extract spatial features of the object image (the image containing the object), and the output feature map retains the spatial structure information of the object image. The feature mapping module can be implemented based on a fully connected network. The module may also be referred to as a fully connected mapping unit module. The function of this module is to map the feature map containing spatial structure information extracted by the deep network unit module into a feature vector of a set dimension, for example, a 1×n dThe vector μ of dimension is the feature vector of the image, that is, the object feature vector (feature representation of the object) of the object contained in the image.

[0108] Step S120: using the first training set to train each first initial feature extraction module respectively, to obtain n trained first object feature extraction modules.

[0109] Since the first training set is a labeled training dataset, supervised training can be performed on each of the first initial feature extraction models based on the first training set to obtain a trained first object feature extraction module. It is understood that the first object feature extraction module here is a feature extraction module obtained by continuously training the first initial feature extraction module.

[0110] The embodiment of the present application does not limit the training method of each first initial feature extraction module based on the first training set, and it can be implemented using an existing training method. Optionally, the identity of the object in the first sample image represented by the label (the real identity category), when training each first initial feature extraction module, a classification module (such as a softmax layer) can be cascaded after the first initial feature extraction module. During the training process, after the first sample image is input into the first initial feature extraction module, the classification module can predict the identity of the first sample image based on the object feature vector extracted by the first initial feature extraction model (for example, the probability that the image in the first sample object belongs to each identity category), and the training loss can be calculated based on the difference between the predicted identity and the real identity.

[0111] When training is performed based on the first training set, the first training set can be divided into multiple batches of data, and each training (that is, one adjustment of the model parameters) uses a batch of data. Each training session inputs a batch (a batch) of first sample images into the first initial feature extraction module. The total training loss corresponding to the first initial feature extraction module can be obtained based on the training loss corresponding to the batch of images. Whether the training meets the conditions can be judged based on the total training loss or the number of training times. If the total training loss meets the conditions or the number of training times reaches the set number, the first initial feature extraction module obtained by training at this time can be used as the first object feature extraction module.

[0112] By continuously training the first initial feature extraction modules of the n first neural network models respectively, n trained first object feature extraction modules can be obtained.

[0113] Step S130: Obtain a second training set and a second neural network model to be trained, where the second training set includes a plurality of second sample images containing sample objects, and the second neural network model includes a second initial feature extraction module.

[0114] The second sample image may be a labeled image or an unlabeled image. The first training set and the second training set may have at least some of the same sample images, or they may be different. The type of sample objects contained in the first sample image and the second sample image may be determined by actual application requirements. For example, if the trained target object feature extraction module is used to extract facial features, then the sample objects should also be faces.

[0115] The second initial feature extraction module is also a feature extraction module to be trained, and the specific model structure of the second initial feature extraction module is not limited in the embodiment of the present application. Optionally, the first initial feature extraction module and the second initial feature extraction module are feature extraction modules based on the same model structure, but compared to the first initial feature extraction module, the second initial feature extraction module can be a lighter-weight feature extraction model with fewer parameters than the first initial feature extraction module. The first neural network model can be called a large recognition network unit module, and the second neural network model can be called a small recognition network unit module.

[0116] Optionally, the second initial feature extraction module may also include a cascaded feature map extraction module and a feature mapping module. Similarly, the feature map extraction module is used to extract the feature map of the input image, and the feature mapping module is used to map the feature map to a specified feature space to obtain a feature vector of the object in the image.

[0117] Step S140: Repeat the first training operation on the second neural network model based on the second training set until the first training end condition is met, thereby obtaining a trained object feature extraction model (that is, a trained second neural network model).

[0118] Among them, the trained object feature extraction model can be used as the feature extraction module of the target recognition model. The target recognition model can be used to extract features from the image containing the object to be identified that is input into the model, and predict the recognition result of the object to be identified based on the extracted object feature vector.

[0119] The first training operation may include: Figure 2 Steps S141 to S144 shown in FIG.

[0120] Step S141: for each second sample image, input the image into n first object feature extraction modules to obtain n first object feature vectors of the image;

[0121] Step S142: For each second sample image, input the image into the second initial feature extraction module to obtain the second object feature vector of the image.

[0122] Step S143: for each second sample image, determining a first training loss corresponding to the image based on the difference between the n first object feature vectors and the second object feature vector of the image;

[0123] Step S144: Based on the first training loss corresponding to each second sample image, determine the first total training loss corresponding to the second neural network model; if the first training end condition is not met, adjust the model parameters of the second neural network model based on the first total training loss.

[0124] Optionally, in actual applications, when training the second initial feature extraction model based on the second training set, the second training set can be divided into multiple batches, with each iterative training using the training data within a batch (i.e., the second sample avatars). The calculation of the first training total loss corresponding to each training is based on the second training loss corresponding to all sample images within the batch. It is understandable that when training the second initial feature extraction module based on multiple batches of training data, each second sample object mentioned above during each training operation refers to each second sample object in the batch used in the current training operation.

[0125] Since the n first object feature extraction modules are already trained feature extraction modules, these n first object feature extraction modules can be used as a teacher network to guide the training of the second initial feature extraction module. The second initial feature extraction module is the network to be trained. For each second sample object, the second object feature vector extracted by the second initial feature extraction module is not accurate enough. This module needs to be learned based on the sample image. The n first object feature vectors extracted by the trained n first object feature extraction modules should have object features with relatively good feature expression capabilities. Therefore, based on the difference between these n first object feature vectors and the second object feature vector extracted by the second initial feature extraction module, the deviation between the second object feature vector and the n first object feature vectors can be calculated, thereby obtaining the training loss corresponding to the sample image. After obtaining the training loss corresponding to each second sample image, the training losses corresponding to each second sample image can be fused (e.g., summed or averaged) to obtain the first total training loss corresponding to the second initial feature extraction module. This total loss can be used to measure the performance of the current neural network model and constrain the training of the module. By continuously performing the above training process until the first training end condition is met, a trained first initial feature extraction model, that is, a first object feature extraction model, can be obtained.

[0126] Among them, the first training end condition can be configured according to actual application requirements, and the embodiment of the present application is not limited to it. It may include but is not limited to the number of training times reaching a set number or the first training total loss meeting a preset condition, which is commonly referred to as the convergence of the training function. For example, the first training total loss is less than a set value or the total training losses corresponding to the set number of consecutive training operations are all less than the set value.

[0127] When training the second neural network model, if the above-mentioned first training end condition is not met, the model parameters (network parameters) of the module can be adjusted, i.e., the parameters can be adjusted. For example, the model parameters can be adjusted based on the gradient descent algorithm, and the adjusted second neural network model can continue to be trained. Through continuous iterative training until the above-mentioned first training end condition is met, the second neural network model that meets the condition can be used as a trained object feature extraction model. The object feature extraction model can be used as a feature extraction module of a target recognition model for object recognition of images containing objects.

[0128] The above-mentioned training method provided in the embodiment of the present application can first obtain multiple first object feature extraction modules based on the first training set with labels. On this basis, the multiple first object feature extraction modules can be used as guidance models to guide the training of the second neural network model to be trained based on the second training set. Since the multiple first object feature extraction modules have been trained, when the multiple first object feature extraction modules are used as guidance models to guide the training of the second neural network model, the finally trained neural network model (that is, the above-mentioned object feature extraction model) can integrate the model capabilities of the multiple first neural network models, thereby effectively improving the performance of the finally obtained object feature extraction model, and better meeting the actual application requirements. When the object feature extraction model is used for image feature extraction, the expression ability of the extracted features can be improved, thereby effectively improving the accuracy of the recognition results when performing object recognition based on features.

[0129] When training the second neural network model, the specific method for calculating the training loss corresponding to each second sample image is not limited in this embodiment of the present application. In theory, any calculation method that can reflect the difference between the n first object feature vectors and the second object feature vector of the second sample image can be used. For example, the difference between each first object feature vector and the second object feature vector can be calculated separately (for example, 1 minus the similarity between the two vectors), the differences corresponding to the n first object feature vectors are averaged, and the average is used as the first training loss.

[0130] In practical applications, the loss function (objective function) corresponding to the second initial feature extraction model can be selected according to needs. By using this function, the first training loss can be obtained based on the difference between the n first object feature vectors and the second object feature vectors of the image.

[0131] Optionally, for each second sample image, determining the first training loss corresponding to the image based on the difference between the n first object feature vectors and the second object feature vector of the image may include:

[0132] The n first object feature vectors corresponding to the image are fused to obtain a first fused feature vector; and a first training loss corresponding to the image is determined based on a difference between the first fused feature vector and the second object feature vector corresponding to the image.

[0133] Among them, the specific method of fusing the n first object feature vectors is not limited in the embodiment of the present application. For example, the n first object feature vectors can be averaged to obtain a first fused feature vector, and then the difference between the first fused feature vector and the second object feature vector is calculated, and the difference is used as the first training loss.

[0134] For each sample image, the n first object feature vectors corresponding to the image may be obtained in the following manner:

[0135] For each first object feature extraction module, feature extraction is performed on the image by the module to obtain a first feature map, and feature mapping is performed on the first feature map to obtain a first object feature vector of the image corresponding to the module.

[0136] The present invention does not limit the model structure used for feature map extraction in the first object feature extraction module and the second initial feature extraction module, and the model structure used for feature mapping of the feature map. Optionally, the feature map (i.e., feature map) can be extracted using a CNN-based model structure, and the feature mapping can be based on a fully connected layer model structure.

[0137] In practical applications, when extracting features from objects in an image, the object feature vectors extracted by the feature extraction module are uncertain due to the presence of occlusion, shadows, or other noise in the image, or other factors (such as the performance of the feature extraction module itself). This uncertainty will affect the effectiveness of the trained target feature extraction module. Taking this into account, the present application also provides an optional solution, in which, for each second sample image, the above-mentioned training operation of the second initial feature extraction module may further include:

[0138] For each first object feature vector of the image, determining a first confidence level of the feature vector based on a first feature map corresponding to the feature vector;

[0139] Accordingly, for each second sample image, determining the first training loss corresponding to the image based on the difference between the n first object feature vectors and the second object feature vector of the image may include:

[0140] For each first object feature vector corresponding to the image, weighting the feature vector using the first confidence level of the feature vector to obtain a weighted first object feature vector;

[0141] fusing the weighted n first object feature vectors corresponding to the image to obtain a second fused feature vector;

[0142] A first training loss corresponding to the image is determined according to a difference between the second fusion feature vector corresponding to the image and the second object feature vector.

[0143] For each of the n first object feature vectors, the confidence of the feature vector represents the degree of trustworthiness of the feature vector, which can also be understood as the accuracy of the feature vector's ability to express the corresponding object. The lower the uncertainty of the feature vector, the higher the corresponding confidence. Since the feature vector is obtained based on the feature map corresponding to the feature vector, the first confidence of the feature vector can be determined based on the feature map corresponding to the feature vector. After determining the first confidence of the n first object feature vectors, the first confidence can be used to weight the corresponding first object feature vector to obtain a weighted first object feature vector with better expression ability. Therefore, the first training loss corresponding to the second sample image can be determined based on the difference between the n weighted first object feature vectors and the second object feature vector. Since the weighted first object feature vector is more accurate, the training loss calculated using the weighted first object feature vector can be used to train the second initial feature extraction model, which can further improve the model performance of the trained target feature extraction model.

[0144] For each second sample image, when determining the first training loss based on the difference between the n weighted first object feature vectors and the second object feature vector corresponding to the image, the first training loss can be obtained by fusing the n weighted first object feature vectors to obtain a second fused feature vector, and then calculating the difference between the second fused feature vector and the second object feature vector to obtain the first training loss corresponding to the image. The method for fusing the n weighted first object feature vectors can be configured according to actual needs and is not limited in the embodiments of the present application.

[0145] As an optional manner, for each second sample image, fusing the weighted n first object feature vectors corresponding to the image to obtain a second fused feature vector may include:

[0146] The second fused feature vector is obtained by the following expression:

[0147]

[0148] in, represents the second fused feature vector, n represents the number of the first neural network models, that is, the number of the first object feature extraction models, μ i Indicates that the second sample image corresponds to the first object feature vector of the i-th first neural network model, 1≤i≤n, k i represents μ i The first confidence level of , || ||2 is the L2 norm calculation symbol.

[0149] Through this optional method, multiple first object feature vectors can be integrated based on the first object feature vectors extracted by multiple first object feature extraction modules and the first confidence of each vector to obtain a second fused feature vector that integrates the capabilities of each first object feature extraction module.

[0150] The specific method for determining the confidence level of an object feature vector is not limited in this embodiment, and can be determined by statistical analysis based on a large number of sample images. As an alternative, the confidence level can be determined by prediction using a neural network model. In this embodiment, the neural network model that predicts the confidence level is referred to as an uncertainty estimation module.

[0151] Optionally, each first neural network model further includes a first uncertainty estimation module to be trained corresponding to the first initial feature extraction module. After obtaining the trained n first object feature extraction modules, the training method may further include:

[0152] Obtaining a third training set, the third training set including a plurality of sample subsets and a category feature vector of each sample subset, each sample subset including at least one third sample image of a sample object, the category feature vector representing a true feature vector of the sample object corresponding to the corresponding sample subset;

[0153] For each first uncertainty estimation module to be trained, repeatedly perform the following second training operation on the uncertainty estimation module based on the third training set until the second training end condition is met, thereby obtaining a trained first uncertainty estimation module:

[0154] For each third sample image, extract features of the image using the first object feature extraction module corresponding to the uncertainty estimation module to obtain a third feature map of the image, obtain a third object feature vector of the image based on the feature map, input the third feature map into the uncertainty estimation module to obtain a third confidence score of the third object feature vector, and determine a second training loss corresponding to the image based on the third object feature vector of the image, the third confidence score, and the category feature vector corresponding to the image;

[0155] Determining a second total training loss corresponding to the uncertainty estimation model based on the second training loss corresponding to each third sample image, and adjusting model parameters of the uncertainty estimation module based on the second total training loss if the second training end condition is not met;

[0156] The first confidence level of each first object feature vector is obtained through a corresponding trained first uncertainty estimation module.

[0157] That is to say, for each first neural network model, the model may include two parts: a feature extraction module and an uncertainty estimation module. These two parts can be trained separately. Specifically, the first initial feature extraction module of the model can be trained based on the first training set. After obtaining the trained first object feature extraction module, the uncertainty estimation module of the model can be trained based on the third training set and the trained first object feature extraction module to obtain a trained uncertainty estimation module, which is used to predict the confidence of the object feature vector extracted by the first object feature extraction module. Then, when the second neural network model is trained based on the trained first neural network model and the second training set, for each second sample image, the first confidence of the first object feature vector extracted by the first object feature extraction module corresponding to the estimation module can be predicted based on the trained uncertainty estimation module.

[0158] Among them, the third training set can be a subset of images corresponding to multiple sample objects, that is, a set of sample images corresponding to each sample object, and the category feature vector (that is, the category center) corresponding to a sample subset represents the label feature of the sample object corresponding to the sample subset, that is, the true feature vector, and the label features corresponding to all third sample images in a sample subset are the same. Then, when training the first uncertainty estimation module to be trained, for each third sample image, the object feature vector, confidence and true feature vector of the image predicted by the first neural network model can be used to calculate the second training loss that can characterize the deviation between the object feature vector predicted by the model and the true feature vector. Among them, the specific method of calculating the second training loss corresponding to each third sample image is not limited in this embodiment of the application, and the corresponding training loss function or objective function can be selected for calculation according to actual needs.

[0159] In actual implementation, since there are multiple first neural network models, each first neural network model has its own model parameters, and the model parameters of different first neural network models are theoretically different, taking this situation into consideration, as an optional solution, for each sample subset, the category feature vector of the sample subset may include the category feature vector of the sample subset corresponding to each first neural network model, that is, there may be n category feature vectors corresponding to each sample subset. For each first neural network model, in the process of training the uncertainty estimation module of the model based on the third training set, when calculating the second training loss corresponding to the third sample image, the category feature vector of the sample image corresponding to the model may be used, that is, for each first uncertainty estimation module, the above-mentioned determination of the second training loss corresponding to each second sample image may be based on the third object feature vector of the second sample image, the third confidence, and the category feature vector of the image corresponding to the first neural network model to which the uncertainty estimation module belongs, to determine the second training loss corresponding to the image.

[0160] The method for obtaining the class feature vector corresponding to each sample subset can be configured according to actual needs. For example, it can be obtained using existing technologies. Optionally, the embodiment of the present application provides two solutions for obtaining the class feature vector. The class feature vector of each sample subset is obtained by any of the following methods:

[0161] Method 1: For each first neural network model, each third sample image in the sample subset is input into the first object feature extraction module of the model to obtain an object feature vector for each third sample image; the object feature vectors of each third sample image in the sample subset are fused to obtain a category feature vector of the sample subset corresponding to the first neural network model;

[0162] Method 2: For each first neural network model, the model parameters corresponding to the category of the sample subset in the model parameters of the first object feature extraction module of the model are used as the category feature vector of the sample subset corresponding to the first neural network model, wherein each sample subset corresponds to one category.

[0163] For method one, since the first object feature extraction module of each first neural network model has been trained, this module can be used to extract object features from each sample image in each sample subset. Since all third sample images in a sample subset are images of the same sample object, the category feature vector representing the sample object can be obtained by fusing the object feature vectors of these sample images.

[0164] For the second method, since the first object feature extraction module of each first neural network model is used to extract feature vectors that can characterize the objects contained in the image, each object can be understood as a category, and the model parameters of the first object feature extraction module include model parameters corresponding to each category (i.e., classification weight parameters of each category). The model parameters of each category can be used as the category center of a category. Therefore, the model parameters of the category corresponding to each appearance subset can be used as the category feature vector. Among them, the first object feature extraction module can include a feature map extraction module and a feature mapping module. The feature mapping module can be a fully connected module. The model parameters of the last fully connected layer of the fully connected module (i.e., the weight matrix) include model parameters of each category. Each category corresponds to an object, and the weight of each category in the model parameters of the feature mapping module can be used as the category feature vector corresponding to the category.

[0165] After obtaining each trained first object feature extraction module based on the first training set, the uncertainty estimation module corresponding to each first object feature extraction module can be further trained based on the third training set, thereby obtaining n trained first neural network models, each of which includes a first object feature extraction module and an uncertainty estimation model corresponding to the feature extraction module. After obtaining the n trained first neural network models, a second neural network model can be trained based on the second training set and these n first neural network models to obtain a trained second neural network model that meets the application requirements, that is, a trained feature extraction model.

[0166] Similarly, for the second neural network model, the object feature vector extracted by the feature extraction module of the model is also uncertain. Then, in order to further improve the performance of the target feature extraction model finally obtained, the second neural network model may optionally further include a second uncertainty estimation module to be trained, which is used to estimate the confidence of the object feature vector extracted by the feature extraction module of the second neural network model. For each second sample image, the second object feature vector of the image may be obtained by:

[0167] Performing feature extraction on the image using a second initial feature extraction module to obtain a second feature map, and performing feature mapping on the second feature map to obtain a second object feature vector of the image;

[0168] For each second sample image, the first training operation may further include: inputting a second feature map corresponding to the image into a second uncertainty estimation module to be trained to obtain a second confidence level corresponding to a second object feature vector of the image;

[0169] For each second sample image, determining the first training loss corresponding to the image based on the difference between the second fused feature vector corresponding to the image and the second object feature vector includes:

[0170] Determining a fusion confidence of a second fused feature vector corresponding to the image;

[0171] A first training loss corresponding to the image is determined based on a difference between the second fused feature vector and the second object feature vector corresponding to the image, and a difference between the fused confidence and the second confidence corresponding to the image.

[0172] In this solution, when training the second neural network model, the feature extraction module and uncertainty estimation module of the model can be trained simultaneously. Accordingly, the first training loss corresponding to each second sample image can not only consider the difference between the object feature vectors, but also the difference between the fusion confidence of the second fused feature vector and the first confidence and the second confidence corresponding to the image predicted by the second neural network model. The training loss calculated in this way constrains the training of the second neural network model, which can make the confidence between the object feature vector predicted by the second neural network model and the feature vector more accurate.

[0173] For each second sample image, the fusion confidence of the second fusion feature vector corresponding to the image can be determined by any of the following methods:

[0174] Determining a fusion confidence of the second fused feature vector based on the second fused feature vector corresponding to the image;

[0175] The n first confidences corresponding to the image are fused to determine a fusion confidence of a second fused feature vector of the image.

[0176] Since the second fused feature vector is obtained by fusing n weighted first object feature vectors, and the n first confidences are the confidences of the first object feature vector before weighting, the fused confidence of the second fused feature vector can be obtained by fusing (for example, averaging) these n first confidences. In addition, since the second fused feature vector is an object feature vector that has been integrated with n first confidences, the fused confidence can also be determined based on the second fused feature vector.

[0177] Optionally, the fusion confidence corresponding to each second sample image may be determined by the following expression:

[0178]

[0179] in, represents the fusion confidence, n represents the number of the first neural network model, that is, the number of the first object feature vectors corresponding to the second sample object, μ i Indicates that the second sample image corresponds to the first object feature vector of the i-th first neural network model, 1≤i≤n, k i represents μ i The first confidence level of , || ||2 is the L2 norm calculation symbol.

[0180] For each second sample object, when determining the first training loss corresponding to the image based on the difference between the second fused feature vector corresponding to the image and the second object feature vector, and the difference between the fused confidence corresponding to the image and the second confidence, as one approach, the first training loss corresponding to the image can be determined by the following expression:

[0181]

[0182] in, Represents the second fused feature vector corresponding to the image, express The fusion confidence, μ φ (x) represents the second object feature vector of the image, represents μ φ The second confidence level of (x).

[0183] The feature extraction model trained based on the training method provided in the embodiment of the present application (that is, the second neural network model after training) can effectively integrate the advantages of multiple first object feature models, and the performance of the model can be greatly improved compared to the prior art. When the feature extraction of the image to be processed is performed through the model, the extracted feature vector has better feature expression ability. Furthermore, the object feature extraction model trained in the embodiment of the present application can also include a trained uncertainty estimation module, which can evaluate the confidence of the extracted feature vector, so that the recognition accuracy of the object can be further improved based on the extracted object feature vector and the confidence of the vector. For example, the object feature vector can be weighted by the confidence of the object feature vector, and the weighted feature vector can be used as the feature vector for determining the recognition result. For example, the model can be used to extract features from two images containing objects, and the feature vectors and the confidence of the feature vectors corresponding to the two images can be obtained. The similarity of the weighted feature vectors corresponding to the two images can be calculated to determine whether the objects in the two images are the same object.

[0184] Based on the above method provided in the embodiment of the present application, the embodiment of the present application also provides an object recognition method, such as Figure 3 As shown, the identification method may include:

[0185] Step S310: Acquire an image to be identified, wherein the image to be identified includes an object to be identified;

[0186] Step S320: extracting features from the image to be identified by calling an object feature extraction model to obtain an object feature vector of the object to be identified; wherein the object feature extraction model is trained using the training method provided in any optional embodiment of the present application;

[0187] Step S330: Determine a recognition result of the object to be recognized based on the object feature vector of the object to be recognized.

[0188] Optionally, determining the recognition result of the object to be identified based on the object feature vector of the object to be identified may include: calculating the similarity between the object feature vector of the object to be identified and the feature vector of an object with a known identity, and determining whether the object to be identified is the object with the known identity based on the similarity; or, there may be multiple images to be identified, and which images among the multiple images are images of the same object may be determined based on the similarity between the object feature vectors of each image to be identified.

[0189] In an optional embodiment of the present application, the object feature extraction model may include an object feature extraction module and an uncertainty estimation module. The object feature extraction of the image to be identified by calling the object feature extraction model to obtain the object feature vector of the object to be identified may include:

[0190] The feature extraction module of the object feature extraction model extracts features from the image to be identified to obtain a feature map of the image to be identified, and obtains an object feature vector of the object to be identified based on the feature map. Based on the feature map, the uncertainty estimation module determines the confidence of the object feature vector;

[0191] Accordingly, the determining of the recognition result of the object to be recognized based on the object feature vector of the object to be recognized may include: determining the recognition result of the object to be recognized based on the object feature vector of the object to be recognized and the confidence of the object feature vector.

[0192] Optionally, the recognition method may further include: acquiring an image to be compared, wherein the image to be compared includes an object to be compared; obtaining an object feature vector of the object to be compared and a confidence level of the object feature vector using the object feature extraction model;

[0193] The above-mentioned determination of the recognition result of the object to be recognized based on the object feature vector of the object to be recognized may include: determining whether the object to be recognized and the object to be recognized are the same object based on the object feature vector of the object to be compared and the confidence of the object feature vector, and the object feature vector of the object to be recognized and the confidence of the object feature vector.

[0194] Among them, as an optional method, the object feature vector of the object to be compared can be weighted by the confidence of the object feature vector of the object to be compared to obtain a weighted first feature vector, the object feature vector of the object to be identified can be weighted by the confidence of the object feature vector of the object to be identified to obtain a weighted second feature vector, the similarity between the weighted first feature vector and the weighted first feature vector is calculated, and whether the object to be identified and the object to be identified are the same object is determined based on the similarity.

[0195] As another possible method, the similarity between the object to be identified and the object to be identified can be calculated using the following expression, and whether the object to be identified and the object to be identified are the same object can be determined based on the similarity:

[0196]

[0197]

[0198]

[0199] Among them, Sim is the similarity, μi and μ j Represent the object feature vector of the object to be identified and the object feature vector of the object to be compared, κ i μ i The confidence level, κ j μ j The confidence level, is the L2 norm of p, is the Bessel function, r is a hyperparameter, and can be a preset value.

[0200] After Sim is calculated, if the similarity is greater than or equal to the similarity threshold, it can be determined that the object to be identified and the object to be identified are the same object. If the similarity is less than the similarity threshold, it can be determined that the object to be identified and the object to be identified are not the same object.

[0201] In order to better illustrate and understand the solution provided by the embodiment of the present application and its practical value, the method provided by the embodiment of the present application is first described in conjunction with an embodiment of an application scenario.

[0202] Optional, Figure 4 FIG. 1 shows a schematic diagram of the structure of an object recognition system applicable to this embodiment of the scenario, such as Figure 4 As shown in , the object recognition system may include a training server 10, an application server 20, and a terminal device 30, wherein the training server 10 may be used to execute the training method provided in the embodiment of the present application to obtain a trained object feature extraction model, use the trained object extraction model as a feature extraction module of the object recognition model, and deploy the object recognition model to the application server 20. The application server 20 may provide object recognition services based on the object model, and the terminal device 30 may be a terminal device with an image acquisition function. The user may use the terminal device to capture an image of the object to be recognized and send the captured image to be recognized to the application server 20. After receiving the image to be recognized, the application server 20 may call the object recognition model to recognize the image and obtain an object recognition result. Optionally, the object recognition result may be displayed to the user through the terminal device 30, or the recognition result may be sent to other pre-set electronic devices.

[0203] Figure 5 and Figure 6 The following are respectively shown in the embodiment of this scenario: Figure 4 The object recognition system shown in the figure performs object recognition in the following manner: Figure 4 、 Figure 5 and Figure 6 The solution provided in the embodiments of the present application is described.

[0204] like Figure 5As shown, the implementation process in this embodiment can be divided into two phases. The first phase is the module training phase, which can be performed by the training server 10. The second phase is the module deployment phase. In this phase, an object recognition model is constructed based on the trained object feature extraction module and deployed to the application server 20. These four phases are described in detail below.

[0205] Part 1: Module training phase

[0206] This stage can be performed by the training server 10. There are n first neural network models to be trained. Each first neural network model includes a first initial feature extraction model to be trained and a first uncertainty estimation module. Figure 5 As shown in Figure 5 The deep network unit module and the fully connected mapping module (that is, the fully connected mapping unit module later) are the first initial feature extraction module, the teacher model uncertainty estimation module is the first uncertainty estimation module, and the small model is the second neural network model to be trained.

[0207] This training phase mainly trains multiple complementary deep network unit modules, fully connected mapping modules, and uncertainty estimation modules to determine the parameters in their structure (i.e., model parameters). In this phase, multiple first neural network models are trained first, and then the second neural network model is trained. Figure 6 As shown in , the training steps in this stage may include steps S11 to S13.

[0208] Step S11: Obtain n first object feature extraction modules based on the first training set training.

[0209] This step is to train the feature extraction modules (including the deep network unit module and the fully connected mapping module) of the n first neural network models respectively to obtain the trained n first object feature extraction models. The training process of this step is as follows: Figure 7a As shown, the process is as follows:

[0210] (a) Training data preparation module: During the training process, this module reads the training data (the first sample object in the first training set), combines the read data into a batch, and sends it to the deep network unit module for processing.

[0211] (b) Deep Network Unit Module: This module extracts spatial features from images containing objects, and the output feature map preserves the spatial structure of the object image. During the training phase, the model extracts features from the first sample input image to generate the corresponding feature map.

[0212] (c) Fully connected mapping unit module: The function of this module is to map the feature map containing spatial structure information extracted by the deep network unit module into a 1×n d A vector of dimension μ (the feature vector of the first sample image).

[0213] (d) Objective function calculation module: This module is used to calculate the training loss of the first initial feature extraction module (deep network unit module and fully connected mapping module). Specifically, it can be based on the feature vector μ output by the fully connected mapping unit to determine the object prediction result corresponding to the first sample image, that is, the predicted recognition result. For example, the classification result f corresponding to μ can be predicted through the softmax layer. The result f can be a probability that the object in the first sample image belongs to each category. The result f and the label of the first sample portrait that produces the result f (the real recognition result) can be used as the input of the objective function calculation module to calculate the objective function value (which is the total training loss). The objective function can be a classification function (such as a softmax loss function, various types of softmax loss functions with margin), or other types of objective functions.

[0214] If the training termination condition corresponding to the first initial feature extraction module is not met ( Figure 7a If the basic model training conditions in are not met, then go to (e).

[0215] (e) Objective function optimization module, which optimizes the training of the deep network unit module and the fully connected mapping module based on the gradient descent method (such as stochastic gradient descent, stochastic gradient descent with momentum, adam, adagard), that is, adjusts the model parameters of these two modules, and then repeats (a) to (d) until the training results meet the training termination conditions. If the termination conditions are not met, the model parameters are adjusted based on the gradient descent method, and the training is repeated. The conditions for terminating model training are generally set to meet the set value when the number of iterations meets the set value, or the loss calculated by the objective function is less than the set value to complete the model training ( Figure 7a The deep network unit module shown in is optimized), and a trained deep network unit module and a fully connected mapping unit module are obtained, which is the first object feature extraction module.

[0216] In the solution provided in the embodiment of the present application, it is necessary to adopt a multi-model integration technology. Therefore, in the training stage, it is necessary to train multiple larger deep network unit modules and their corresponding fully connected mapping modules, that is, to train multiple first object feature extraction modules. In implementation, the model structures of multiple first initial feature extraction modules to be trained can be the same, but the model parameters are different. Optionally, a basic feature extraction network can be constructed first, and multiple different first initial feature extraction modules can be obtained by changing the randomly initialized seed of the basic feature extraction network module, that is, multiple structures including the above-mentioned deep network unit modules and fully connected mapping unit modules with different initialized model parameters). Then, after the above-mentioned training process (a) to (e), multiple trained first object feature extraction models are obtained, and each first object feature extraction model includes a trained deep network unit module and a fully connected mapping unit module.

[0217] like Figure 8 As shown, in the process of model integration, multiple first object feature extraction modules can be used to extract features from the same image. The features extracted by each model are distributed around the real features. The integration method can be used to obtain more accurate image features, that is, the feature vector of the object in the image. Figure 8 As shown in the figure, the value of n is 6. The feature vectors extracted by the six first object feature extraction modules are Feature1 to Feature6 in the figure. Ensemble Feature represents the fusion of Feature1 to Feature6. The fused feature vector obtained has better object representation ability than the individual features in Feature1 to Feature6.

[0218] After obtaining n trained first object feature extraction models, you can execute Figure 6 In step S12 shown in , the first uncertainty estimation modules corresponding to the n first object feature extraction modules are obtained through training based on the third training set.

[0219] In the process of training the uncertainty estimation module, it is necessary to use Figure 7a In the process of training the uncertainty estimation module, the parameters of the deep network unit module and the fully connected mapping unit module are not updated, that is, the model parameters of the trained first object feature extraction module are fixed, and each first object feature extraction module (which can be called a large model) trains its own paired uncertainty estimation module.

[0220] The training implementation process of step S12 is as follows: Figure 7bAs shown, the process is as follows:

[0221] (a) Training data preparation module. The function of this module is consistent with the training data preparation module in the deep network unit module training, and is used to obtain the third training set in this stage.

[0222] (b) Training sample class center acquisition module, the function of this module is to calculate the class center w of the sample image (sample subset) in each ID (one ID corresponds to one sample object) in the third training set x∈c , w x∈c Represents the class center of the c-th class sample, that is, the class feature vector corresponding to the sample object in a sample subset. The calculation method of this module can be to take the mean operation of all sample image features in a class (such as those extracted by the first object feature module), or the classification weight of each class obtained by training the deep network unit module can be used as its class center.

[0223] (c) An uncertainty estimation module, the function of which is to estimate the uncertainty k (confidence in this scenario embodiment) of the feature vector of each input third sample image in the hypersphere space. The structure of this module may include but is not limited to a model based on a CNN structure (such as: a neural network in the form of RESNET with several layers of full connections). For each third sample image, the third feature map corresponding to the image can be extracted by a trained deep network unit module, and the feature map is input into the uncertainty estimation module to obtain the confidence k corresponding to the feature map, and the feature map is input into the trained fully connected mapping unit module to obtain the third object feature vector μ(x) corresponding to the image.

[0224] (d) Uncertainty objective function calculation module, which is used to calculate the feature μ(x) obtained by the fully connected mapping unit module and the class center w of the category. x∈c Calculate the objective function, that is, calculate the second training loss. For each third sample image, the second training loss L corresponding to the image can be calculated by the calculation formula shown in formula (1): s , where k represents the uncertainty or confidence of the image feature (object feature vector), μ(x) represents the output feature of the fully connected mapping unit module, d represents the feature dimension output by the fully connected mapping module, r represents the radius of the feature mapping to the hypersphere space, and I is the Bessel function, the expression of which is shown in (2).

[0225]

[0226]

[0227] (e) Uncertainty objective function optimization module, which optimizes the entire uncertainty estimation module based on the gradient descent method (such as stochastic gradient descent, stochastic gradient descent with momentum term, adam, adagard), that is, adjusts the model parameters of the uncertainty estimation model. The optimized gradients for k and μ are shown in formula (3) and formula (4). Repeat (a) to (d) during training until the training results meet the training termination condition (the second training end condition). The condition for terminating model training is generally set to the number of iterations meeting the set value, or the loss calculated by the objective function (the second training total loss, obtained based on the second training loss corresponding to the third sample image used in each iterative training) is less than the set value to complete the model training and obtain the trained first uncertainty estimation module.

[0228]

[0229]

[0230] After obtaining the trained n first neural network models through steps S11 and S12, the small model can be trained based on the second training set and the n first neural network models to obtain the final object feature extraction model.

[0231] Step S13: training the second neural network model based on the n trained first neural network models and the second training set to obtain an object feature extraction model.

[0232] This step is to use an integrated approach to distill the small model. The functions of each module and the training process involved in this step are as follows: Figure 9 As shown, in this step, the parts that need to update the model parameters are the modules with background filling in the figure, namely the small recognition network unit module and the small uncertainty estimation module. Among them, the small recognition network unit module is the second initial feature extraction module of the second neural network model, and the small uncertainty estimation module is the second uncertainty estimation module of the second neural network model. Figure 9 The large recognition network unit module in the training is the first object feature extraction module of the first neural network model, and the uncertainty estimation module 1 to the uncertainty estimation module n respectively represent the first uncertainty estimation modules corresponding to the n trained first neural network models. Figure 9 As shown, the modules and processes involved in this step can be as follows:

[0233] (a) Training data preparation module. The function of this module is consistent with the training data preparation module in the previous deep network unit module training. It is used to obtain a second training set, which includes a large number of second sample images.

[0234] (b) Large recognition network unit module. There are n modules in this module, which are integrated, so there are multiple large networks. This is the large recognition network (first object feature extraction module) trained in step S11. In this step, the large recognition network serves as a teacher network, and its model parameters are not updated.

[0235] (c) Uncertainty estimation module, which also has n modules and is the uncertainty estimation model trained in step S12. This model is matched one-to-one with each large recognition network unit module, and its model parameters are not updated.

[0236] (d) Integrated feature and uncertainty calculation module, which integrates the features and uncertainty estimates extracted by n large recognition networks and outputs the integrated features and uncertainty The integration formula is shown in formula (5-1) and formula (5-2), where μ1~μ n and κ1~κ n is the uncertainty of the features and correspondence of each model to the image, μ1~μ n Corresponding respectively Figure 9 fea1~fean, κ1~κ in n Corresponding respectively Figure 9 kappa1~kappa n in.

[0237]

[0238]

[0239] In step S13, for each second sample image, the image can be input into n large recognition network unit modules respectively, and the feature map of the image can be extracted by the large recognition network unit module, and the corresponding feature μ (that is, the object feature vector) can be obtained based on the feature map. Based on the feature map, the uncertainty estimation module matched with the large recognition network unit module can be used to predict the uncertainty k of the feature μ. Further, the fused feature can be obtained based on the above formulas (5-1) and (5-2). and uncertainty That is the second fused feature vector and fused confidence mentioned above.

[0240] (e) Small recognition network unit module and small uncertainty estimation module. In this step, the structure of the small recognition network unit module can be consistent with that of the feature extraction module in step S11, with a small number of parameters. During the training phase, the input of this module is the object image (the second sample image), and the output is the extracted feature u (the second object feature vector, shown as Fea in the figure) and the intermediate layer feature map m (the second feature map). The structure of the small uncertainty estimation module can be consistent with that of the first uncertainty estimation module in step S12, with the input being the feature map m and the output being the uncertainty kappa (the second confidence level).

[0241] (f) Knowledge distillation objective function calculation module: This module is used to calculate the second training loss corresponding to each second sample image, and then obtain the total training loss based on the second training loss of each second sample image. The input of this module is the integrated feature of the large model. Uncertainty And the output feature μ of the small network φ (x) Uncertainty The formula for calculating the loss can be shown as formula (6).

[0242]

[0243] in, represents the training loss corresponding to a second sample image, Indicates that the goal of model training is to minimize the total training loss, such as less than the set value.

[0244] (g) Knowledge distillation objective function optimization module. This module optimizes the small network recognition module and the corresponding uncertainty estimation module based on the gradient descent method (such as stochastic gradient descent, stochastic gradient descent with momentum term, adam, adagard), that is, adjusts the model parameters of the second neural network model. By repeating the above (a) to (d), until the training result meets the training termination condition (the first training end condition). The condition for terminating model training is generally set to the number of iterations meeting the set value, or the loss calculated by the mu objective function calculation module (the first training total loss) is less than the set value to complete the model training (that is, the knowledge distillation training optimization is completed), and a trained object feature extraction model (including the trained small recognition network unit module and the small uncertainty estimation module) is obtained.

[0245] like Figure 5 and Figure 6 As shown in , after completing the training of the small model, the module deployment phase can be entered. This phase is to build a complete object recognition model based on the trained small model, and the object recognition model can be deployed on the application server 20 for identifying objects in the image. Figure 5 The deep network unit module and the fully connected mapping module in the module deployment stage are the trained object feature extraction modules in the small model, and the uncertainty estimation module is the trained second uncertainty estimation module in the small model.

[0246] The module deployment stage mainly combines and deploys the relevant modules obtained in the module training stage to form a complete solution. As an optional solution, the object recognition model can be used to determine whether the objects in the two images to be compared are the same object. The recognition model can include a deep network unit module, a fully connected mapping unit module and an uncertainty estimation module of a trained small model. It can also include a distance measurement unit module and a threshold judgment unit module. Among them, the deep network unit module is used to extract the feature map of the image to be recognized input into the object recognition model, the fully connected mapping unit module obtains the feature vector of the image based on the feature map mapping, the uncertainty estimation module predicts the uncertainty of the feature vector based on the feature map, the distance measurement unit module calculates the similarity between the two images based on the feature vectors and corresponding uncertainties of the two images to be compared, and the threshold judgment unit module determines whether the objects in the two images are the same object based on the similarity output by the distance measurement unit module.

[0247] The application process at this stage can be as follows Figure 10 As shown, the two images to be compared are input into the deep network unit module, which outputs a feature map with high expression information. The feature map passes through the fully connected mapping module to obtain the feature μ of dimension d, i.e., the feature vector. At the same time, the feature map passes through the uncertainty estimation module to obtain the uncertainty estimation factor k corresponding to the image, i.e., the confidence. i and μ j ) and uncertainty estimation factor (k i and k j ) and enter the distance measurement module at the same time, which calculates the similarity of the image pair through formulas (7)-(9) to obtain the similarity sim of the compared image pair.

[0248]

[0249]

[0250]

[0251] After calculating the similarity sim, the threshold judgment unit module judges whether the two images are from the same person based on the image similarity output by the similarity calculation module (i.e., the distance measurement module). The threshold th can be determined based on the false alarm rate required to be met in the actual application scenario. For example, a reference data set can be obtained, and the reference data set includes multiple sample image pairs. Each sample image pair can be input into the object recognition model to obtain the prediction results of each sample image pair (whether the objects in the predicted sample image pairs are the same object). According to the prediction results of all sample image pairs and the false alarm rate to be met, the corresponding threshold can be determined. The output L of the threshold judgment unit module is out As shown in formula (10):

[0252]

[0253] As an example of practical application, the application server may pre-collect and store images of multiple objects with known identities. One of the two images to be compared may be an image of an object to be identified collected in real time by the terminal device 30. Each image pre-stored in the application server may be used as another image to be compared. Figure 10 The identification process shown in , determines the identity of the object to be identified. Figure 5 As shown in FIG, the terminal device 30 sends the image of the object to be identified collected in real time to the application server 30 through steps S21 and S22, and the application server 30 calls the object recognition model and executes Figure 10 The process shown (step S31) determines which object in the image to be identified is pre-stored in the server, or verifies whether the object in the image to be identified is one of the objects in the pre-stored image, and can send the recognition result to the terminal device 30 (step S32). For example, if the object is identified as an object with a known identity, then the recognition result can be a matching identity or identity authentication passed.

[0254] The solution provided by the embodiment of the present application combines the uncertainty analysis of image features (i.e., the feature vector of the object obtained based on image extraction) with the integrated model, and uses the integrated model to obtain a more accurate uncertainty estimate of the features corresponding to the image. It is then used in the distillation training of the small model to improve the recognition accuracy of the small model. In the training stage, multiple network models with complementary deep network unit modules and fully connected mapping unit modules for object recognition can be trained first. Then, the parameters of the deep network unit module and the fully connected mapping unit module in each network model are output as the input of each paired uncertainty estimation module to train the uncertainty estimation module. In this process, only the model parameters of the uncertainty estimation module are trained, and the parameters in the deep network unit module and the fully connected mapping unit module are not updated. Afterwards, the trained deep network unit module is integrated with the respective uncertainty estimation modules to perform distillation training on the small model. After obtaining the trained deep network unit module, fully connected mapping unit module and uncertainty estimation module of the small model, these modules in the small model are integrated in the module deployment stage, and can be combined with the distance measurement module and the threshold judgment unit module to form a complete object recognition model. By adopting the solution provided in the embodiments of the present application, the recognition accuracy of the object recognition system can be improved, its false recognition alarm rate can be reduced, and the robustness of the small recognition model in complex scenarios can be increased.

[0255] Corresponding to the training method provided in the embodiment of the present application, the embodiment of the present application also provides a training device for an object feature extraction model, such as Figure 11 As shown, the training device 100 may include a training data acquisition module 110 , a first model training module 120 and a second model training module 130 .

[0256] A training data acquisition module 110 is configured to acquire a first training set and n first neural network models to be trained, where n ≥ 2, wherein the first training set includes a plurality of first sample images with labels, the labels representing actual recognition results of sample objects contained in the first sample images, and the first neural network model includes a first initial feature extraction module; and to acquire a second training set and a second neural network model to be trained, wherein the second training set includes a plurality of second sample images containing sample objects, and the second neural network model includes a second initial feature extraction module;

[0257] A first model training module 120 is configured to train each first initial feature extraction module using a first training set to obtain n trained first object feature extraction modules;

[0258] The second model training module 130 is configured to repeatedly perform the following first training operation on the second neural network model until the first training end condition is met to obtain a trained object feature extraction model. The first training operation includes:

[0259] For each second sample image, the image is input into n first object feature extraction modules to obtain n first object feature vectors of the image, and the image is input into the second initial feature extraction module to obtain the second object feature vector of the image. Based on the difference between the n first object feature vectors and the second object feature vector of the image, the first training loss corresponding to the image is determined; based on the first training loss corresponding to each second sample image, the first total training loss corresponding to the second initial feature extraction module is determined. If the first training end condition is not met, the model parameters of the second neural network model are adjusted based on the first total training loss.

[0260] Optionally, for each second sample image, the second model training module may be configured to:

[0261] The n first object feature vectors corresponding to the image are fused to obtain a first fused feature vector; and a first training loss corresponding to the image is determined based on a difference between the first fused feature vector and the second object feature vector corresponding to the image.

[0262] Optionally, for each second sample image, the n first object feature vectors corresponding to the image are obtained in the following manner:

[0263] For each first object feature extraction module, extract features from the image using the module to obtain a first feature map, and perform feature mapping on the first feature map to obtain a first object feature vector corresponding to the module;

[0264] For each second sample image, the second model training module may be used to:

[0265] For each first object feature vector of the image, determine a first confidence of the feature vector based on a first feature map corresponding to the feature vector; for each first object feature vector corresponding to the image, weight the feature vector using the first confidence of the feature vector to obtain a weighted first object feature vector; fuse the n weighted first object feature vectors corresponding to the image to obtain a second fused feature vector; and determine a first training loss corresponding to the image based on a difference between the second fused feature vector and the second object feature vector corresponding to the image.

[0266] Optionally, each first neural network model further includes a first uncertainty estimation module to be trained corresponding to the first initial feature extraction module. After obtaining the trained n first object feature extraction modules, the first model training module may further be used to perform the following operations:

[0267] Obtaining a third training set, the third training set including a plurality of sample subsets and a category feature vector of each sample subset, each sample subset including at least one third sample image of a sample object, the category feature vector representing a true feature vector of the sample object corresponding to the corresponding sample subset;

[0268] For each first uncertainty estimation module to be trained, repeatedly performing the second training operation on the uncertainty estimation module based on the third training set until the second training end condition is met, thereby obtaining a trained first uncertainty estimation module. The second training operation includes:

[0269] For each third sample image, the third feature map of the image is obtained by the first object feature extraction module corresponding to the uncertainty estimation module, and the third object feature vector of the image is obtained based on the feature map, the third feature map is input into the uncertainty estimation module to obtain the third confidence of the third object feature vector, and the second training loss corresponding to the image is determined based on the third object feature vector, the third confidence, and the category feature vector corresponding to the image; based on the second training loss corresponding to each third sample image, the second total training loss corresponding to the uncertainty estimation model is determined, and if the second training end condition is not met, the model parameters of the uncertainty estimation module are adjusted based on the second total training loss; wherein, the first confidence of each first object feature vector is obtained by the corresponding trained first uncertainty estimation module.

[0270] Optionally, for each second sample image, the second object feature vector of the image is obtained in the following manner:

[0271] Performing feature extraction on the image using a second initial feature extraction module to obtain a second feature map, and performing feature mapping on the second feature map to obtain a second object feature vector of the image;

[0272] The second neural network model also includes a second uncertainty estimation module to be trained. For each second sample image, the first training operation further includes:

[0273] Inputting a second feature map corresponding to the image into a second uncertainty estimation module to be trained to obtain a second confidence level corresponding to a second object feature vector of the image;

[0274] For each second sample image, the second model training module may be used to:

[0275] Determine a fusion confidence of a second fusion feature vector corresponding to the image; and determine a first training loss corresponding to the image based on a difference between the second fusion feature vector corresponding to the image and the second object feature vector, and a difference between the fusion confidence corresponding to the image and the second confidence.

[0276] Optionally, for each second sample image, the second model training module may determine the fusion confidence of the second fused feature vector corresponding to the image by any one of the following:

[0277] Determining a fusion confidence of the second fused feature vector based on the second fused feature vector corresponding to the image;

[0278] The n first confidences corresponding to the image are fused to determine a fusion confidence of a second fused feature vector of the image.

[0279] Optionally, for each second sample image, when the second model training module fuses the weighted n first object feature vectors corresponding to the image to obtain a second fused feature vector, the second fused feature vector can be obtained by the following expression:

[0280]

[0281] in, represents the second fusion feature vector, n represents the number of the first neural network model, μ i Indicates that the second sample image corresponds to the first object feature vector of the i-th first neural network model, 1≤i≤n, k i represents μ i The first confidence level of , || ||2 is the L2 norm calculation symbol.

[0282] Optionally, the fusion confidence corresponding to each second sample image is determined by the following expression:

[0283]

[0284] in, represents the fusion confidence, n represents the number of the first neural network model, μ i Indicates that the second sample image corresponds to the first object feature vector of the i-th first neural network model, 1≤i≤n, k i represents μ i The first confidence level of , || ||2 is the L2 norm calculation symbol.

[0285] Optionally, the first training loss corresponding to each second sample image is determined by the following expression:

[0286]

[0287] in, Represents the second fused feature vector corresponding to the image, express The fusion confidence, μ φ (x) represents the second object feature vector of the image, represents μ φ The second confidence level of (x).

[0288] Optionally, for each sample subset, the category feature vector of the sample subset includes the category feature vector of each first neural network model corresponding to the sample subset;

[0289] For each first uncertainty estimation module, when the first model training module determines the second training loss corresponding to each second sample image based on the third object feature vector, the third confidence level, and the category feature vector corresponding to the image, the first model training module may be used to:

[0290] Determine a second training loss corresponding to the image based on a third object feature vector of the image, a third confidence level, and a category feature vector of the image corresponding to the first neural network model to which the uncertainty estimation module belongs;

[0291] The category feature vector of each sample subset can be obtained by any of the following methods:

[0292] For each first neural network model, each third sample image in the sample subset is input into the first object feature extraction module of the model to obtain an object feature vector of each third sample image; the object feature vectors of each third sample image in the sample subset are fused to obtain a category feature vector of the sample subset corresponding to the first neural network model;

[0293] For each first neural network model, the model parameters corresponding to the category of the sample subset in the model parameters of the first object feature extraction module of the model are used as the category feature vector of the sample subset corresponding to the first neural network model, wherein each sample subset corresponds to one category.

[0294] Corresponding to the object recognition method provided in the embodiment of the present application, the embodiment of the present application also provides an object recognition device, such as Figure 12 As shown, the object recognition device 200 may include a to-be-processed image acquisition module 210 and an image recognition module 220 .

[0295] The image to be processed acquisition module 110 is used to acquire an image to be identified, wherein the image to be identified includes an object to be identified;

[0296] The image recognition module 120 is configured to extract features from the image to be recognized by calling an object feature extraction model to obtain an object feature vector of the object to be recognized, and determine a recognition result of the object to be recognized based on the object feature vector of the object to be recognized;

[0297] The object feature extraction model is obtained by training using any training method provided in any optional embodiment of the present application.

[0298] Optionally, the image recognition module can be used to:

[0299] Extracting features of the image to be identified by the object feature extraction module to obtain a feature map of the image to be identified, and obtaining an object feature vector of the object to be identified and a confidence level of the object feature vector based on the feature map;

[0300] A recognition result of the object to be recognized is determined based on the object feature vector of the object to be recognized and the confidence level of the object feature vector.

[0301] Optionally, the module for acquiring images to be processed may also be used for:

[0302] Acquire an image to be compared, wherein the image to be compared includes an object to be compared; obtain an object feature vector of the object to be compared and a confidence level of the object feature vector using the object feature extraction model;

[0303] The image recognition module can be used to determine whether the object to be identified and the object to be identified are the same object based on the object feature vector of the object to be compared and the confidence of the object feature vector, and the object feature vector of the object to be identified and the confidence of the object feature vector.

[0304] It can be understood that the device of the embodiment of the present application can execute the method provided by the embodiment of the present application, and the implementation principle is similar. The actions performed by each module in the device of each embodiment of the present application correspond to the steps in the method of each embodiment of the present application. For the detailed functional description of each module of the device, please refer to the description in the corresponding method shown in the previous text, and will not be repeated here.

[0305] An embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory. When the processor executes the computer program stored in the memory, the method in any optional embodiment of the present application can be implemented.

[0306] Figure 13FIG. 1 shows a schematic structural diagram of an electronic device to which an embodiment of the present invention is applicable. Figure 13 As shown, the electronic device may be a server or a user terminal, and the electronic device may be used to implement the method provided in any embodiment of the present invention.

[0307] like Figure 13 As shown in FIG, the electronic device 2000 may mainly include at least one processor 2001 ( Figure 13 1 ), memory 2002, communication module 2003 and input / output interface 2004 and other components, optionally, the components can be connected and communicated through bus 2005. It should be noted that, Figure 13 The structure of the electronic device 2000 shown in the figure is merely illustrative and does not constitute a limitation on the electronic devices to which the method provided in the embodiments of the present application is applicable.

[0308] Memory 2002 can be used to store operating systems and application programs, etc. Application programs can include computer programs that implement the methods described in the embodiments of the present invention when called by processor 2001, and can also include programs for implementing other functions or services. Memory 2002 can be ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices that can store information and computer programs, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0309] The processor 2001 is connected to the memory 2002 via the bus 2005 and implements corresponding functions by calling the application program stored in the memory 2002. The processor 2001 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof, which can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. The processor 2001 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0310] The electronic device 2000 can be connected to a network via a communication module 2003 (which may include, but is not limited to, components such as a network interface) to communicate with other devices (such as a user terminal or a server) via the network to implement data interaction, such as sending data to or receiving data from other devices. The communication module 2003 may include a wired network interface and / or a wireless network interface, etc., that is, the communication module may include at least one of a wired communication module and a wireless communication module.

[0311] The electronic device 2000 can be connected to the required input / output devices, such as a keyboard, a display device, etc., through the input / output interface 2004. The electronic device 2000 itself can have a display device, and can also be connected to other external display devices through the interface 2004. Optionally, a storage device, such as a hard disk, can also be connected through the interface 2004, so that data in the electronic device 2000 can be stored in the storage device, or data in the storage device can be read, and data in the storage device can also be stored in the memory 2002. It can be understood that the input / output interface 2004 can be a wired interface or a wireless interface. Depending on the actual application scenario, the device connected to the input / output interface 2004 can be a component of the electronic device 2000, or it can be an external device connected to the electronic device 2000 when needed.

[0312] Bus 2005, used to connect the various components, may include a path for transmitting information between the components. Bus 2005 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. Depending on their function, bus 2005 may be categorized as an address bus, a data bus, a control bus, or the like.

[0313] Optionally, for the solution provided in the embodiment of the present invention, the memory 2002 can be used to store a computer program for executing the solution of the present invention, and be run by the processor 2001. When the processor 2001 runs the computer program, the actions of the method or device provided in the embodiment of the present invention are implemented.

[0314] Based on the same principle as the method provided in the embodiment of the present application, the embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the corresponding content of the aforementioned method embodiment can be implemented.

[0315] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the corresponding content of the aforementioned method embodiment can be implemented.

[0316] It should be noted that the terms "first," "second," "third," "fourth," "1," "2," etc. (if any) in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described in the drawings.

[0317] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the order of implementation of these steps is not limited to the order indicated by the arrows. Unless otherwise clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart can include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage in these sub-steps or stages can also be executed at different times respectively. Under different scenarios at the execution time, the execution order of these sub-steps or stages can be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0318] The above description is only an optional implementation method for some implementation scenarios of this application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of this application, the use of other similar implementation methods based on the technical ideas of this application also falls within the protection scope of the embodiments of this application.

Claims

1. A method for training a feature extraction model, characterized in that: include: Obtaining a first training set and n first neural network models to be trained, where n is greater than or equal to 2, wherein the first training set includes a plurality of first sample images with labels, wherein the labels represent actual recognition results of sample objects contained in the first sample images, and the first neural network model includes a first initial feature extraction module; Using the first training set, training each first initial feature extraction module separately to obtain n trained first object feature extraction modules; Obtaining a second training set and a second neural network model to be trained, wherein the second training set includes a plurality of second sample images containing sample objects, and the second neural network model includes a second initial feature extraction module; Repeating the first training operation on the second neural network model until the first training end condition is met to obtain a trained object feature extraction model, wherein the first training operation includes: For each second sample image, input the image into the n first object feature extraction modules to obtain n first object feature vectors of the image, input the image into the second initial feature extraction module to obtain a second object feature vector of the image, fuse the n first object feature vectors corresponding to the image to obtain a fused feature vector corresponding to the image, and determine a first training loss corresponding to the image based on a difference between the fused feature vector and the second object feature vector corresponding to the image; Based on the first training loss corresponding to each of the second sample images, the first total training loss corresponding to the second neural network model is determined; if the first training end condition is not met, the model parameters of the second neural network model are adjusted based on the first total training loss.

2. The method according to claim 1, characterized in that For each of the second sample images, the n first object feature vectors corresponding to the image are obtained in the following manner: For each of the first object feature extraction modules, extract features from the image using the module to obtain a first feature map, and perform feature mapping on the first feature map to obtain a first object feature vector corresponding to the module; For each of the second sample images, the first training operation further includes: For each first object feature vector of the image, determining a first confidence level of the feature vector based on a first feature map corresponding to the feature vector; For each second sample image, fusing the n first object feature vectors corresponding to the image to obtain a fused feature vector corresponding to the image, including: For each first object feature vector corresponding to the image, weighting the feature vector using the first confidence level of the feature vector to obtain a weighted first object feature vector; The weighted n first object feature vectors corresponding to the image are fused to obtain a fused feature vector corresponding to the image.

3. The method according to claim 2, characterized in that Each of the first neural network models further includes a first uncertainty estimation module to be trained corresponding to the first initial feature extraction module. After obtaining the trained n first object feature extraction modules, the method further includes: Obtaining a third training set, the third training set comprising a plurality of sample subsets and a class feature vector of each sample subset, each sample subset comprising at least one third sample image of a sample object, the class feature vector representing a true feature vector of the sample object corresponding to the corresponding sample subset; For each first uncertainty estimation module to be trained, repeatedly performing a second training operation on the uncertainty estimation module based on the third training set until a second training end condition is met, thereby obtaining a trained first uncertainty estimation module, wherein the second training operation includes: For each of the third sample images, obtaining a third feature map of the image through the first object feature extraction module corresponding to the uncertainty estimation module, and obtaining a third object feature vector of the image based on the feature map, inputting the third feature map into the uncertainty estimation module to obtain a third confidence score of the third object feature vector, and determining a second training loss corresponding to the image based on the third object feature vector of the image, the third confidence score, and the category feature vector corresponding to the image; determining a second total training loss corresponding to the uncertainty estimation module based on the second training loss corresponding to each of the third sample images, and adjusting model parameters of the uncertainty estimation module based on the second total training loss if the second training end condition is not met; The first confidence of each first object feature vector is obtained through a corresponding trained first uncertainty estimation module.

4. The method according to claim 3, characterized in that For each of the second sample images, the second object feature vector of the image is obtained by: Performing feature extraction on the image by the second initial feature extraction module to obtain a second feature map, and performing feature mapping on the second feature map to obtain a second object feature vector of the image; The second neural network model further includes a second uncertainty estimation module to be trained. For each second sample image, the first training operation further includes: Inputting the second feature map corresponding to the image into the second uncertainty estimation module to be trained to obtain a second confidence level corresponding to the second object feature vector of the image; For each second sample image, determining the first training loss corresponding to the image based on the difference between the fused feature vector corresponding to the image and the second object feature vector includes: Determine the fusion confidence of the fusion feature vector corresponding to the image; A first training loss corresponding to the image is determined according to a difference between the fusion feature vector corresponding to the image and the second object feature vector, and a difference between the fusion confidence corresponding to the image and the second confidence.

5. The method according to claim 4, characterized in that For each second sample image, determining the fusion confidence of the fusion feature vector corresponding to the image includes any one of the following: Determining a fusion confidence of the fused feature vector based on the fused feature vector corresponding to the image; The n first confidences corresponding to the image are fused to obtain a fusion confidence of the fused feature vector corresponding to the image.

6. The method according to claim 2, characterized in that The fusing the weighted n first object feature vectors corresponding to the image to obtain a fused feature vector corresponding to the image includes: The fused feature vector is obtained by the following expression: in, represents the fused feature vector, n represents the number of the first neural network model, Indicates that the second sample image corresponds to the i The first object feature vector of the first neural network model, 1≤ i ≤ n , express The first confidence level, Computes the sign for the L2 norm.

7. The method according to claim 4 or 5, characterized in that The fusion confidence corresponding to each second sample image is determined by the following expression: in, represents the fusion confidence, n represents the number of the first neural network model, Indicates that the second sample image corresponds to the i The first object feature vector of the first neural network model, 1≤ i ≤ n , express The first confidence level, Computes the sign for the L2 norm.

8. The method according to claim 4 or 5, characterized in that The first training loss corresponding to each second sample image is determined by the following expression: in, Represents the fused feature vector corresponding to the image, express The fusion confidence of represents the second object feature vector of the image, express The second confidence level.

9. The method according to claim 3, characterized in that For each of the sample subsets, the category feature vector of the sample subset includes the category feature vector of each first neural network model corresponding to the sample subset; For each first uncertainty estimation module, determining a second training loss corresponding to the image based on the third object feature vector of the image, the third confidence level, and the category feature vector corresponding to the image includes: Determine a second training loss corresponding to the image based on a third object feature vector of the image, a third confidence level, and a category feature vector of the image corresponding to the first neural network model to which the uncertainty estimation module belongs; The category feature vector of each sample subset is obtained by any of the following methods: For each of the first neural network models, inputting each third sample image in the sample subset into the first object feature extraction module of the model to obtain an object feature vector for each third sample image; fusing the object feature vectors of the third sample images in the sample subset to obtain a category feature vector of the sample subset corresponding to the first neural network model; For each of the first neural network models, the model parameters corresponding to the category of the sample subset in the model parameters of the first object feature extraction module of the model are used as the category feature vector of the sample subset corresponding to the first neural network model, wherein each of the sample subsets corresponds to a category.

10. An object recognition method, characterized in that: include: Acquire an image to be identified, wherein the image to be identified includes an object to be identified; Extracting features from the image to be identified by calling an object feature extraction model to obtain an object feature vector of the object to be identified; Determining a recognition result of the object to be recognized based on the object feature vector of the object to be recognized; Wherein, the object feature extraction model is trained using the method described in any one of claims 1 to 9.

11. The method according to claim 10, characterized in that The object feature extraction module is trained using the method described in claim 4, and extracting features from the image to be identified by calling the object feature extraction model to obtain the object feature vector of the object to be identified includes: Extracting features of the image to be identified by the object feature extraction module to obtain a feature map of the image to be identified, and obtaining an object feature vector of the object to be identified and a confidence level of the object feature vector based on the feature map; The determining the recognition result of the object to be recognized based on the object feature vector of the object to be recognized includes: A recognition result of the object to be recognized is determined based on the object feature vector of the object to be recognized and the confidence level of the object feature vector.

12. The method according to claim 11, characterized in that Also includes: Acquire an image to be compared, wherein the image to be compared includes an object to be compared; Obtaining an object feature vector of the object to be compared and a confidence level of the object feature vector through the object feature extraction model; The determining of the recognition result of the object to be recognized based on the object feature vector of the object to be recognized includes: Based on the object feature vector of the object to be compared and the confidence of the object feature vector, and the object feature vector of the object to be identified and the confidence of the object feature vector, it is determined whether the object to be identified and the object to be compared are the same object.

13. The method according to claim 12, characterized in that The determining whether the object to be identified and the object to be compared are the same object includes: weighting the object feature vector of the object to be compared using the confidence of the object feature vector of the object to be compared to obtain a first feature vector; weighting the object feature vector of the object to be identified using the confidence level of the object feature vector of the object to be identified to obtain a second feature vector; According to the similarity between the first feature vector and the second feature vector, it is determined whether the object to be identified and the object to be compared are the same object.

14. A training device for an object feature extraction model, characterized in that: include: a training data acquisition module, configured to acquire a first training set and n first neural network models to be trained, where n ≥ 2, wherein the first training set includes a plurality of first sample images with labels, wherein the labels represent actual recognition results of sample objects contained in the first sample images, and the first neural network model includes a first initial feature extraction module; and for obtaining a second training set and a second neural network model to be trained, wherein the second training set includes a plurality of second sample images containing sample objects, and the second neural network model includes a second initial feature extraction module; A first model training module is configured to train each first initial feature extraction module using the first training set to obtain n trained first object feature extraction modules; The second model training module is configured to repeatedly perform a first training operation on the second neural network model until a first training end condition is satisfied, thereby obtaining a trained object feature extraction model. The first training operation includes: For each second sample image, input the image into the n first object feature extraction modules to obtain n first object feature vectors of the image, input the image into the second initial feature extraction module to obtain a second object feature vector of the image, fuse the n first object feature vectors corresponding to the image to obtain a fused feature vector corresponding to the image, and determine a first training loss corresponding to the image based on a difference between the fused feature vector and the second object feature vector corresponding to the image; Based on the first training loss corresponding to each of the second sample images, the first total training loss corresponding to the second neural network model is determined; if the first training end condition is not met, the model parameters of the second neural network model are adjusted based on the first total training loss.

15. The device according to claim 14, characterized in that For each of the second sample images, the n first object feature vectors corresponding to the image are obtained in the following manner: For each of the first object feature extraction modules, extract features from the image using the module to obtain a first feature map, and perform feature mapping on the first feature map to obtain a first object feature vector corresponding to the module; For each of the second sample images, the first training operation further includes: For each first object feature vector of the image, determining a first confidence level of the feature vector based on a first feature map corresponding to the feature vector; For each of the second sample images, the fused feature vector corresponding to the image is obtained in the following manner: For each first object feature vector corresponding to the image, weighting the feature vector using the first confidence level of the feature vector to obtain a weighted first object feature vector; The weighted n first object feature vectors corresponding to the image are fused to obtain a fused feature vector corresponding to the image.

16. The device according to claim 15, characterized in that Each of the first neural network models further includes a first uncertainty estimation module to be trained corresponding to the first initial feature extraction module. After obtaining the trained n first object feature extraction modules, the first model training module is further configured to perform the following operations: Obtaining a third training set, the third training set comprising a plurality of sample subsets and a class feature vector of each sample subset, each sample subset comprising at least one third sample image of a sample object, the class feature vector representing a true feature vector of the sample object corresponding to the corresponding sample subset; For each first uncertainty estimation module to be trained, repeatedly performing a second training operation on the uncertainty estimation module based on the third training set until a second training end condition is met, thereby obtaining a trained first uncertainty estimation module, wherein the second training operation includes: For each of the third sample images, obtaining a third feature map of the image through the first object feature extraction module corresponding to the uncertainty estimation module, and obtaining a third object feature vector of the image based on the feature map, inputting the third feature map into the uncertainty estimation module to obtain a third confidence score of the third object feature vector, and determining a second training loss corresponding to the image based on the third object feature vector of the image, the third confidence score, and the category feature vector corresponding to the image; determining a second total training loss corresponding to the uncertainty estimation module based on the second training loss corresponding to each of the third sample images, and adjusting model parameters of the uncertainty estimation module based on the second total training loss if the second training end condition is not met; The first confidence of each first object feature vector is obtained through a corresponding trained first uncertainty estimation module.

17. The device according to claim 16, characterized in that For each of the second sample images, the second object feature vector of the image is obtained by: Performing feature extraction on the image by the second initial feature extraction module to obtain a second feature map, and performing feature mapping on the second feature map to obtain a second object feature vector of the image; The second neural network model further includes a second uncertainty estimation module to be trained. For each second sample image, the first training operation further includes: Inputting the second feature map corresponding to the image into the second uncertainty estimation module to be trained to obtain a second confidence level corresponding to the second object feature vector of the image; For each of the second sample images, the first training loss corresponding to the image is determined by: Determine the fusion confidence of the fusion feature vector corresponding to the image; A first training loss corresponding to the image is determined according to a difference between the fusion feature vector corresponding to the image and the second object feature vector, and a difference between the fusion confidence corresponding to the image and the second confidence.

18. The device according to claim 17, characterized in that For each of the second sample images, the fusion confidence of the fusion feature vector corresponding to the image is determined by any one of the following: Determining a fusion confidence of the fused feature vector based on the fused feature vector corresponding to the image; The n first confidences corresponding to the image are fused to obtain a fusion confidence of the fused feature vector corresponding to the image.

19. The device according to claim 15, characterized in that The fused feature vector corresponding to the image is obtained by the following expression: in, represents the fused feature vector, n represents the number of the first neural network model, Indicates that the second sample image corresponds to the i The first object feature vector of the first neural network model, 1≤ i ≤ n , express The first confidence level, Computes the sign for the L2 norm.

20. The device according to claim 17 or 18, characterized in that The fusion confidence corresponding to each second sample image is determined by the following expression: in, represents the fusion confidence, n represents the number of the first neural network model, Indicates that the second sample image corresponds to the i The first object feature vector of the first neural network model, 1≤ i ≤ n , express The first confidence level, Computes the sign for the L2 norm.

21. The device according to claim 17 or 18, characterized in that The first training loss corresponding to each second sample image is determined by the following expression: in, Represents the fused feature vector corresponding to the image, express The fusion confidence of represents the second object feature vector of the image, express The second confidence level.

22. The device according to claim 16, characterized in that For each of the sample subsets, the category feature vector of the sample subset includes the category feature vector of each first neural network model corresponding to the sample subset; For each first uncertainty estimation module, determining a second training loss corresponding to the image based on the third object feature vector of the image, the third confidence level, and the category feature vector corresponding to the image includes: Determine a second training loss corresponding to the image based on a third object feature vector of the image, a third confidence level, and a category feature vector of the image corresponding to the first neural network model to which the uncertainty estimation module belongs; The category feature vector of each sample subset is obtained by any of the following methods: For each of the first neural network models, inputting each third sample image in the sample subset into the first object feature extraction module of the model to obtain an object feature vector for each third sample image; fusing the object feature vectors of the third sample images in the sample subset to obtain a category feature vector of the sample subset corresponding to the first neural network model; For each of the first neural network models, the model parameters corresponding to the category of the sample subset in the model parameters of the first object feature extraction module of the model are used as the category feature vector of the sample subset corresponding to the first neural network model, wherein each of the sample subsets corresponds to a category.

23. An object recognition device, characterized in that: include: The module for acquiring an image to be processed is used to acquire an image to be identified, wherein the image to be identified includes an object to be identified; An image recognition module is configured to extract features from the image to be recognized by calling an object feature extraction model to obtain an object feature vector of the object to be recognized, and determine a recognition result of the object to be recognized based on the object feature vector of the object to be recognized; Wherein, the object feature extraction model is trained using the method described in any one of claims 1 to 9.

24. The device according to claim 23, characterized in that The object feature extraction module is trained using the method described in claim 4, and the object feature vector of the object to be identified is obtained by: Extracting features of the image to be identified by the object feature extraction module to obtain a feature map of the image to be identified, and obtaining an object feature vector of the object to be identified and a confidence level of the object feature vector based on the feature map; The identification result of the object to be identified is determined by: A recognition result of the object to be recognized is determined based on the object feature vector of the object to be recognized and the confidence level of the object feature vector.

25. The device according to claim 24, characterized in that The to-be-processed image acquisition module is further configured to: acquire an image to be compared, wherein the image to be compared includes an object to be compared; The image recognition module is further configured to: obtain an object feature vector of the object to be compared and a confidence level of the object feature vector through the object feature extraction model; Wherein, when determining the recognition result of the object to be recognized based on the object feature vector of the object to be recognized and the confidence level of the object feature vector, the image recognition module is used to: Based on the object feature vector of the object to be compared and the confidence of the object feature vector, and the object feature vector of the object to be identified and the confidence of the object feature vector, it is determined whether the object to be identified and the object to be compared are the same object.

26. The device according to claim 25, characterized in that The image recognition module is used to determine whether the object to be recognized and the object to be compared are the same object: weighting the object feature vector of the object to be compared using the confidence of the object feature vector of the object to be compared to obtain a first feature vector; weighting the object feature vector of the object to be identified using the confidence level of the object feature vector of the object to be identified to obtain a second feature vector; According to the similarity between the first feature vector and the second feature vector, it is determined whether the object to be identified and the object to be compared are the same object.

27. An electronic device, characterized in that: The electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the method according to any one of claims 1 to 9 or the method according to any one of claims 10 to 13 by executing the computer program.

28. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 or the method according to any one of claims 10 to 13 is implemented.

29. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 or the method according to any one of claims 10 to 13 is implemented.

Images