Object recognition method, network training method, apparatus, device, and medium

Abstract

Embodiments of the present application provide an object recognition method, a network training method, an apparatus, a device and a medium, wherein the method comprises: obtaining at least two image domains of a picture including a sample object; determining a global threshold for evaluating the similarity of the sample object in the at least two image domains and a local threshold for evaluating the similarity of the sample object in each image domain based on a preset misrecognition rate; adjusting the classification loss of each image domain in a to-be-trained object recognition network based on the global threshold and the local threshold of each image domain to obtain a first loss; and adjusting the network parameters of the to-be-trained object recognition network based on the first loss and a second loss of the at least two image domains, so that the loss output by the adjusted object recognition network satisfies a preset convergence condition.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and relate to but are not limited to an object recognition method, a network training method, an apparatus, a device, and a medium. BACKGROUND

[0002] Face comparison is an important technology in the field of computer vision. Face comparison can be applied in scenarios such as mobile phone unlocking, gate access, and access control clocking. In related technologies, the evaluation protocol of an object recognition network usually uses different thresholds to calculate the correct acceptance rate (TAR) of different data sets under a pre-defined false acceptance rate (FAR); this does not conform to actual applications, resulting in low performance and fairness of the object recognition network. SUMMARY

[0003] Embodiments of the present application provide a technical solution for training an object recognition network.

[0004] The technical solution of the embodiments of the present application is implemented as follows:

[0005] In a first aspect, the embodiments of the present application provide a method for training an object recognition network, the method comprising:

[0006] obtaining a picture comprising at least two image domains of a sample object;

[0007] determining a global threshold for evaluating the similarity of the sample object in the at least two image domains and a local threshold for evaluating the similarity of the sample object in each image domain based on a pre-set false acceptance rate;

[0008] adjusting the classification loss of each image domain in a to-be-trained object recognition network based on the global threshold and the local threshold of each image domain to obtain a first loss;

[0009] adjusting the network parameters of the to-be-trained object recognition network based on the first loss and a second loss of the at least two image domains, so that the loss output by the adjusted object recognition network satisfies a pre-set convergence condition.

[0010] In a second aspect, the embodiments of the present application provide an object recognition method, the method comprising:

[0011] obtaining a to-be-recognized image of a to-be-recognized object in a picture;

[0012] performing feature extraction on the to-be-recognized image based on an object recognition network to obtain image features; wherein the object recognition network is trained based on the method of the first aspect;

[0013] Based on the image feature, a target preset image in which a marked object in the picture matches the to-be-identified object is searched in a preset image library.

[0014] In a third aspect, an embodiment of the present application provides a device for training an object recognition network, and the device comprises:

[0015] A first acquisition module is configured to acquire at least two image domains in which a picture includes a sample object.

[0016] A first determination module is configured to determine, based on a preset misrecognition rate, a global threshold for evaluating the similarity of the sample object in the at least two image domains and a local threshold for evaluating the similarity of the sample object in each image domain.

[0017] A first adjustment module is configured to adjust, based on the global threshold and the local threshold of each image domain, a classification loss of each image domain in a to-be-trained object recognition network to obtain a first loss.

[0018] A second adjustment module is configured to adjust, based on the first loss and a second loss of the at least two image domains, a network parameter of the to-be-trained object recognition network, so that an adjusted object recognition network outputs a loss that meets a preset convergence condition.

[0019] In a fourth aspect, an embodiment of the present application provides an object recognition device, and the device comprises:

[0020] A third acquisition module is configured to acquire a to-be-identified image including a to-be-identified object.

[0021] A first extraction module is configured to perform feature extraction on the to-be-identified image based on an object recognition network to obtain an image feature, wherein the object recognition network is trained based on the method of the first aspect.

[0022] A first search module is configured to search, based on the image feature, a target preset image in which a marked object in the picture matches the to-be-identified object in a preset image library.

[0023] An embodiment of the present application provides a computer storage medium, and the computer storage medium stores computer executable instructions. After the computer executable instructions are executed, the object recognition network training method or the object recognition method described above can be implemented.

[0024] An embodiment of the present application provides a computer device, and the computer device comprises a memory and a processor. The memory stores computer executable instructions, and the processor executes the computer executable instructions stored in the memory to implement the object recognition network training method or the object recognition method described above.

[0025] The embodiment of the present application provides a computer program product, which comprises a non-transitory computer-readable storage medium storing a computer program, and the computer program is read and executed by a computer to implement part or all of the steps in the above method.

[0026] The embodiment of the present application provides an object recognition method, a network training method, device and medium. For a sample image set including a sample object, first, in a to-be-trained object recognition network, multiple frames of sample images are divided into multiple image domains according to the sample object; in this way, the multiple frames of sample images are divided into multiple domain sample images, so that the performance of the to-be-trained object recognition network in the multiple image domains can be analyzed. Secondly, in the process of dividing the sample image set into multiple image domains, the second loss of the to-be-trained object recognition network is determined by analyzing the probability of the sample image belonging to each image domain; in this way, the second loss can represent the difference between the prediction probability of the sample object in each image domain and the image domain to which the sample image belongs, so that the second loss can make the image domains divided by the to-be-trained object recognition network more accurate. Thirdly, under a preset misrecognition rate, a global threshold representing the similarity of objects in at least two image domains and a local threshold representing the similarity of objects in each image domain are determined; in this way, for multiple image domains, the global threshold commonly used by the multiple image domains and the local threshold of each image domain are determined according to the similarity between sample objects in the multiple image domains under the same preset misrecognition rate; so that the classification loss of the sample image in the image domain in the to-be-trained object recognition network is adjusted by the global threshold and the local threshold of each image domain to obtain the first loss, so that the first loss can dynamically change with the local threshold of each image domain, and the consistency between the local thresholds of different image domains is improved. Finally, the network parameters of the to-be-trained object recognition network are adjusted by fusing the second loss and the first loss, and the training process of the to-be-trained object recognition network is realized. In this way, by dividing the sample image set into multiple image domains, the classification loss of the sample image in the image domain can be dynamically adjusted according to the global threshold and the local threshold of each image domain, and then the to-be-trained object recognition network is trained by using the dynamically adjusted first loss and second loss, which can improve the performance of the trained object recognition network and improve the accuracy of the object recognition network in multiple image domains. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 The implementation flowchart of the object recognition network training method provided by the embodiment of the present application is shown in the figure.

[0028] Figure 2 Another implementation flowchart of the object recognition network training method provided by the embodiment of the present application is shown in the figure.

[0029] Figure 3 Another implementation flow diagram of the object recognition method provided by the embodiments of the present application is shown in FIG. 9.

[0030] Figure 4 A similarity distribution diagram of two data sets provided by the related embodiments is shown in FIG. 10.

[0031] Figure 5 A TCP implementation framework diagram provided by the embodiments of the present application is shown in FIG. 11.

[0032] Figure 6 A visualization result diagram of domain division in IDD provided by the embodiments of the present application is shown in FIG. 12.

[0033] Figure 7 A visualization result diagram of threshold fairness of different methods in the training process provided by the embodiments of the present application is shown in FIG. 13.

[0034] Figure 8 A structure composition diagram of the training device of the object recognition network provided by the embodiments of the present application is shown in FIG. 14.

[0035] Figure 9 A structure composition diagram of the object recognition device provided by the embodiments of the present application is shown in FIG. 15.

[0036] Figure 10 A composition structure diagram of the computer device provided by the embodiments of the present application is shown in FIG. 16. DETAILED DESCRIPTION

[0037] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the specific technical solutions of the present application will be further described in combination with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but not to limit the scope of the present application.

[0038] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subset of all possible embodiments, and can be combined with each other without conflict.

[0039] In the following description, the terms "first\second\third" are only to distinguish similar objects, and do not represent a specific order of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0041] Before the embodiments of the present application are further described, the terms and names involved in the embodiments of the present application are explained, and the terms and names involved in the embodiments of the present application are applicable to the following explanations.

[0042] 1) Object recognition is a biometric technology based on facial feature information for identity recognition. It uses a camera or a camera to collect images or video streams containing human faces, and automatically detects and tracks faces in images, and then performs a series of related technologies for face recognition on the detected faces, which is also commonly known as portrait recognition, face recognition.

[0043] 2) Computer vision refers to using a camera and a computer to replace the human eye to identify, track and measure the target machine vision, and further process the image to make the computer processing more suitable for human eye observation or image transmission to instrument detection.

[0044] The following describes an exemplary application of the device for training the object recognition network provided by the embodiments of the present application. The device provided by the embodiments of the present application can be implemented as various types of terminals with image acquisition functions, such as notebook computers, tablet computers, desktop computers, cameras, mobile devices (for example, personal digital assistants, smart wearable devices, portable game devices), and servers. In the following, exemplary applications when the device is implemented as a computer device or a server will be described.

[0045] Taking the computer device as an example, the object recognition network training method can be applied to the computer device, and the functions realized by the method can be realized by calling program codes by the processor in the computer device. Of course, the program codes can be saved in the computer storage medium. Therefore, the computer device at least includes a processor and a storage medium.

[0046] The embodiments of the present application provide an object recognition network training method, as shown in Figure 1 , which is described in conjunction with the steps shown in Figure 1 .

[0047] Step S101, at least two image domains including sample objects are acquired.

[0048] In some embodiments, each image domain includes sample images with similar visual information or non-visual information. The device obtains at least one frame of sample images including sample objects, and each frame of sample objects in the at least one frame of sample images can be the same or different. The sample images can be images with complex backgrounds or images with simple backgrounds. The sample objects in the sample images can be labeled with information tags that can roughly describe the sample objects; at least one of the following: the source of the sample objects (for example, in the case of sample objects being personnel, describing the sample objects from which country, which company or from which organization, etc.), the category of the sample objects, etc., and in the case of the at least one frame of sample images being multiple frames, multiple frames of sample images of the same category of sample objects can be taken of the same individual. Taking the case of the sample objects being personnel as an example, multiple frames of sample images of the same category of sample objects can be obtained by image acquisition of the same person.

[0049] Here, the device can obtain the sample images in the following ways: selecting from a sample image library that stores a large number of sample images with labeled sample object attribute information; or labeling images without sample object attribute information. The attribute information of the sample objects can include visual attributes and non-visual attributes of the sample objects.

[0050] In some embodiments, the sample objects can be personnel or animals; the sample images can be face images or full-body images. In the case of the sample objects being personnel, face images of the sample objects are obtained; or in the case of the sample objects being animals, full-body images or face images of the sample objects are obtained. In a specific example, a sample image is a passport photo image, which is a face image of personnel; for another example, a sample image is a video image frame taken by a camera deployed in a zoo for security data acquisition, which is a full-body image of an animal. It should be noted that the at least one frame of images can be obtained from a video, and a video can include multiple video frames, each corresponding to a frame of image. In this way, the device can obtain multiple frames of sample images with labeled attribute information of sample objects.

[0051] In some embodiments, one image domain can be a sample image group, and the sample image group can have multiple frames of sample images with similar visual attributes, such as sample objects with the same face orientation in one image domain, or sample objects with the same skin color in one image domain, or sample objects with the same gender in one image domain, or sample objects with similar facial features in one image domain (for example, in the case of the sample objects being personnel, the personnel in one image domain all have relatively stereoscopic facial features).

[0052] Step S102, determining a global threshold for evaluating the similarity of the sample objects in the at least two image domains and a local threshold for evaluating the similarity of the sample objects in each image domain based on a preset false recognition rate.

[0053] In some embodiments, for each image domain, the global threshold and the local threshold of each image domain are set at the same preset false recognition rate. The preset false recognition rate can be set based on the performance requirement of the object recognition network to be trained, or can be set based on the experience value of the object recognition network to be trained, for example, the preset false recognition rate is set to (1e-4).

[0054] The global threshold is used to evaluate the similarity between the sample objects in multiple images. The global threshold can be set by the similarity of the image features of the multiple image domains and the feature pairs (wherein each feature pair includes one image feature in the image domain and one image feature in the preset feature queue storing the image features of the sample images) formed in the preset feature queue. In some possible implementations, the global threshold can be set by the quantile of the similarity in the feature pairs, so that the global threshold set in this way can match the multiple image domains. The local threshold corresponding to each image domain is used to evaluate the similarity between the sample objects in the image domain. At the preset false recognition rate, for an image domain, the local threshold of the image domain is determined by inputting the number of feature pairs formed in the image domain and the similarity between the features in each feature pair.

[0055] Step S103, adjusting the classification loss of each image domain in the object recognition network to be trained based on the global threshold and the local threshold of each image domain, to obtain a first loss.

[0056] In some embodiments, for each image domain, the local threshold value and the global threshold value of the image domain are combined, and the ratio of the local threshold value and the global threshold value is used as the weight of the image domain. That is, the importance of the image domain in the network training process is described by the ratio between the global threshold value and the local threshold value. The classification loss of each image domain in the object recognition network to be trained is used to represent the difference between the predicted face class and the true value class in the face recognition process of the sample pair in the network training process. In some possible implementation manners, each image domain is input into the object recognition network to be trained, and the face recognition of the sample object in the image domain is performed to predict the difference between the recognition result and the true value. In an embodiment, the type of classification loss can adopt an additive angular margin loss (ArcFace), in which the similarity between the image features of the sample object and the image features of the sample object in other images is represented by the sum of the angles between them; or the type of loss function is multiplicative angular margin loss, in which the similarity between the image features of the sample object and the image features of the sample object in other images is represented by the product of the angles between them; or the loss function is additive cosine margin loss, in which the similarity between the image features of the sample object and the image features of the sample object in other images is represented by the cosine of the angle between them.

[0057] In this way, the ratio between the global threshold value and the local threshold value of each image domain is used as the weight of the classification loss of the image domain, and the classification loss of each image domain can change with the change of the local threshold value of the image domain, so as to dynamically adjust the classification loss of each image domain, and then the weight of the sample image in the image domain in the training process can be dynamically adjusted through the back propagation of the classification loss.

[0058] In step S104, the network parameters of the object recognition network to be trained are adjusted based on the first loss and the second loss of the at least two image domains, so that the loss output by the adjusted object recognition network satisfies a preset convergence condition.

[0059] In some embodiments, the second loss of the object recognition network to be trained is determined based on a probability that the sample image belongs to the at least two image domains. The probability that the sample image belongs to the at least two image domains can be understood as a probability that the sample image is predicted to belong to each image domain. In implementation, if the sample image set is divided into three image domains, the probability that each sample image belongs to each of the three image domains is obtained. In the process of dividing the sample image set into three image domains, the probability that each sample image belongs to each image domain can be analyzed, and the image domain with the highest probability is taken as the image domain to which the sample image belongs. In a specific example, assuming that the first image domain is an image domain in which the skin color of the sample object is close, the second image domain is an image domain in which the face of the sample object is directly facing the collection device, and the third image domain is an image domain in which the sample object has a large face twist angle, the probability that the skin color of the sample object matches the first image domain (for example, the skin color of the sample object is the same as or similar to the skin color of the object in the first image domain), the probability that the face orientation of the sample object is consistent with the second image domain, and the probability that the face twist angle of the sample object is consistent with the third image domain are obtained.

[0060] In some embodiments, the second loss is used to represent the difference between the domain (i.e., the domain label of the sample image) to which the sample image belongs generated in the process of domain division of the sample image and the probability that the sample image belongs to each domain predicted.

[0061] In some possible implementations, for each sample image, the image domain corresponding to the sample image is determined, and the probabilities of each sample image in the image domain are summed up; the second loss is determined by the cross-entropy loss between the maximum probability corresponding to the sample image (which indicates which image domain the sample image is most likely to belong to) and the sum result. In this way, the second loss can effectively reduce the difference between the label of the image domain corresponding to each sample image and the probability that the sample image belongs to the image domain predicted.

[0062] In the iteration process, the network parameters in the object recognition network to be trained are adjusted by combining the first loss and the second loss of the at least two image domains, until the loss of the object recognition network to be trained meets the convergence condition, the iteration process is stopped, the training of the object recognition network to be trained is completed, and the adjusted object recognition network is obtained.

[0063] In the process of training the network by using the loss of the sample objects in multiple image domains, the second loss and the first loss are used to divide the image domains, so that the to-be-trained object recognition network learns the features of the sample objects in different domains, thereby the adjusted object recognition network can extract image features more conducive to cross-image domain recognition; and then in the scene of cross-image domain object recognition, the image features extracted by the adjusted object recognition network can be used to find matching images, thereby the accuracy of cross-image domain object recognition can be improved.

[0064] In the embodiments of the present application, first, a sample image set is obtained, and in the to-be-trained object recognition network, multiple frames of sample images are divided into multiple image domains according to sample objects; in this way, the multiple frames of sample images are divided into multiple domain sample images, so that the performance of the to-be-trained object recognition network in multiple image domains can be analyzed. Secondly, in the process of dividing the sample image set into multiple image domains, the second loss of the to-be-trained object recognition network is determined by analyzing the probability of the sample image belonging to each image domain; in this way, the second loss can represent the difference between the predicted probability of the sample object in each image domain and the image domain to which the sample image belongs, so that the second loss can make the image domains divided by the to-be-trained object recognition network more accurate. Thirdly, under a preset misrecognition rate, a global threshold representing the similarity of objects in at least two image domains and a local threshold representing the similarity of objects in each image domain are determined; in this way, for multiple image domains, the global threshold commonly used by the multiple image domains and the local threshold of each image domain are determined according to the similarity between the sample objects in the multiple image domains under the same preset misrecognition rate; thereby the classification loss of the sample images in the image domain in the to-be-trained object recognition network can be adjusted by the global threshold and the local threshold of each image domain to obtain the first loss, so that the first loss can dynamically change with the local threshold of each image domain, thereby improving the consistency between the local thresholds of different image domains. Finally, the network parameters of the to-be-trained object recognition network are adjusted by fusing the second loss and the first loss, thereby the training process of the to-be-trained object recognition network is realized. In this way, by dividing the sample image set into multiple image domains, the classification loss of the sample images in the image domain can be dynamically adjusted according to the global threshold and the local threshold of each image domain, and then the to-be-trained object recognition network is trained by using the dynamically adjusted first loss and second loss, thereby the performance of the trained object recognition network can be improved, and the fairness of the object recognition network in different image domains can be improved.

[0065] In some embodiments, the obtained sample image set is implicitly segmented to obtain multiple image domains, that is, the above step S101 can be implemented by the following steps S111 and S112 (not shown in the figure):

[0066] Step S111, obtaining a sample image set in which a sample object is labeled with attribute information of the sample object.

[0067] In some embodiments, the visual attribute includes an attribute that can be visualized in a sample image, and the multiple frames of sample images in which the sample object is labeled with the visual attribute and a non-visual attribute are obtained, to obtain the sample image set. For example, the sample object presents a facial picture, skin color, or face orientation in the sample image; the non-visual attribute is an attribute that the sample object has but cannot be visualized in the sample image, such as the origin of the sample object, the organization to which the sample object belongs, or the category of the sample object.

[0068] Step S112, in the to-be-trained object recognition network, the sample image set is divided into the at least two image domains based on the attribute information of the sample object and the multiple preset image domains of the labeled domain labels.

[0069] In some embodiments, the to-be-trained object recognition network can be a network for facial recognition of a to-be-recognized object. The to-be-trained object recognition network is used for feature extraction and feature classification. According to the similarity between the extracted image features and the image features in the image library, a matched multiple frames of images are determined in the image library; the feature extraction can be implemented in the to-be-trained object recognition network by using a convolution layer, and the feature classification can be implemented by using a pooling layer and / or a fully connected layer, and the like. Here, the to-be-trained object recognition network can be an untrained network model; or, it can also be a pre-trained network model.

[0070] In some embodiments, since the sample object is labeled with attribute information in the sample image, in the to-be-trained object recognition network, sample objects with similar attribute information can be predicted according to the attribute information labeled on the sample object, and the sample images in which the sample objects are located are divided into an image domain, so that at least two image domains can be predicted. The sample image set is input into the to-be-trained object recognition network, feature extraction is performed on the sample image set, and the domain label of the sample object is predicted by combining the extracted features with the attribute information of the sample object; in this way, the sample image set is divided into sample images of multiple domains in a self-supervised manner, that is, multiple image domains are obtained, and implicit division of the sample image set is realized.

[0071] In some possible implementations, the multiple preset image domains can be a predefined hyperparameter of an implicit domain quantity, which is irrelevant to the number of image frames in the sample image set. Each image domain in the multiple preset image domains has a domain label, such as a deep skin color, a large face twist angle, a face irrelevant stereoscopic, and the like. In this way, by analyzing the attribute information of the sample object, the probability of the sample object in each image domain can be predicted, and the sample image set is divided into multiple image domains.

[0072] The steps S111 and S112 are used to divide the sample image set into multiple image domains by predicting the image domains to which the sample image set belongs according to the attribute information marked on the sample object, so as to train the object recognition network to be trained by using the multiple image domains, thereby improving the object recognition performance of the object recognition network in the multiple image domains.

[0073] In some embodiments, the extracted features of the sample image are subjected to dimension transformation, so as to analyze the correlation between different second image features, thereby achieving the division of the sample image set in different domains, that is, the step S112 can be implemented by the following steps S1211 to S1213 (not shown in the figure):

[0074] In step S1211, the sample image set is subjected to feature extraction in the object recognition network to be trained, thereby obtaining a first image feature set.

[0075] In some embodiments, in the process of dividing the sample image set into image domains, the sample image set is first input into the object recognition network to be trained, and the features of the sample image are extracted by using the face recognition model in the object recognition network, thereby obtaining the first image feature of each sample image. In some possible implementation manners, the face recognition model can be a multi-task convolutional network used for object recognition by face region detection and face key point detection. In this way, the multi-task convolutional network is used to extract the features of the sample image, thereby obtaining the first image feature set.

[0076] In step S1212, the dimensions of the features in the first image feature set are transformed, thereby obtaining a second image feature set.

[0077] In some embodiments, the features in the first image feature set are converted into high-dimensional features, thereby obtaining the second image features. In some possible implementation manners, after the first image feature set is output from the face recognition model, the first image feature set is input into two concatenated fully connected layers, and the dimensions of the features in the first image feature set are transformed by using the matrix in the fully connected layer, thereby obtaining the second image features.

[0078] In this way, the dimensions of the features in the first image feature set are transformed by using the multiple fully connected layers, thereby obtaining the second image features with unified dimensions and high dimensions, which is more conducive to domain division.

[0079] In step S1213, based on the attribute information of the second image features in the second image feature set, the image domain to which the second image features match is determined in the multiple preset image domains.

[0080] In some embodiments, for any sample image, the attribute information of the second image feature of the sample image is compared with each domain label in terms of similarity, and the higher the similarity is, the greater the probability that the second image feature belongs to the preset image domain is. The preset image domain with the greatest probability of the second image feature is taken as the image domain to which the second image feature is matched. In this way, according to the domain label of the preset image domain, the similarity between the domain label and the attribute information is analyzed, and the matched image domain is predicted for each second image feature,

[0081] In the embodiments of the present application, the dimension transformation is performed on the extracted image features, so that the second image features in the second image feature set are more convenient for subsequent operations; the attribute information of the second image features of each sample image is analyzed, so that the similarity between different second image features is determined, and the image domain to which each sample image belongs is predicted according to the similarity, so that the multiple sample images can be divided into multiple image domains without sample labels, and the dependence on sample image labels is reduced, thereby reducing the calculation cost.

[0082] In some embodiments, the implicit division of the sample images is realized by predicting the probability that each sample image belongs to a preset image domain, and multiple image domains are obtained, that is, the above step S1213 can be realized by the following steps:

[0083] Firstly, based on the attribute information of the second image feature, the probability that the second image feature matches the domain label of each preset image domain is predicted.

[0084] In some embodiments, after the feature dimension of the first image feature set is transformed into the second image feature set, the similarity between the attribute information of different second image features and each preset image domain is predicted according to the attribute information carried by the second image feature. The similarity represents whether the attribute information of the sample object to which the second image feature belongs is consistent with the required attribute information in the preset image domain; for example, the label in the preset image domain is that the face direction of the sample object is directly facing the collection device, and the attribute information is the face direction of the sample object. By analyzing the face direction of the second image feature, the probability that the face direction of the sample object in the sample image is directly facing the collection device can be determined, that is, the probability that the sample image belongs to the preset image domain is obtained.

[0085] Secondly, based on the probability, the image domain to which the sample image to which the second image feature belongs is matched is determined.

[0086] In some embodiments, according to the probability that the attribute information of the second image feature belongs to multiple preset image domains, a preset image domain with a probability greater than a certain probability threshold is taken as the image domain matched by the second image feature, and the sample image to which the second image feature belongs is segmented in the preset image domain. In this way, the sample image set can be segmented into multiple image domains. In some possible implementation manners, the probability threshold can be set according to an empirical value of implicitly dividing image domains, for example, the probability threshold is set to 0.6.

[0087] In some embodiments, the maximum probability can be selected as the image domain matched by the second image feature from the probabilities that the second image feature belongs to multiple preset image domains. In this way, the probabilities of the sample images in each preset image domain are predicted through the attribute information of different second image features, and then the image domain to which each sample image belongs can be labeled through the predicted probabilities, so that implicit division of the sample image set can be quickly realized at low cost.

[0088] In some embodiments, by normalizing the probability that each sample image belongs to a preset image domain, to reduce the difference between the predicted probability and the domain label of the image domain to which the sample image belongs, a cross-entropy loss is taken as the second loss of the object recognition network to be trained, which can be realized by the following steps:

[0089] Firstly, the probabilities that the sample images in the sample image set belong to each preset image domain are normalized to obtain an updated probability set.

[0090] In some embodiments, the probability that each sample image in the sample image set belongs to each preset image domain is obtained. For example, there are five preset image domains, and the probability that each sample image belongs to the five preset image domains is obtained, so that five probabilities are obtained for each sample image. Each probability is expectedly normalized to obtain the prior probability of the probability, that is, the updated probability. In this way, the updated probability is used to represent the confidence that the sample image belongs to each preset image domain, which can reduce the concentration of sample images in certain preset image regions caused by excessively large or small probabilities, and further make the distribution of sample images in different preset image domains more uniform.

[0091] Secondly, a target updated probability satisfying a preset condition is determined from the updated probability set.

[0092] In some embodiments, the preset condition can be set according to the probabilities in the updated probability set, such as taking the maximum probability in the updated probabilities as the target updated probability, or taking the updated probabilities in the updated probability set that are greater than a probability threshold as the target updated probability. The target updated probability can be a probability representing the preset image domain to which the sample image is most likely to belong, that is, the domain label of the preset image domain corresponding to the target updated probability can be considered as the domain label to which the sample image belongs.

[0093] In the third step, the cross-entropy loss between the target updated probability and the probability set is determined as the second loss for segmenting the sample image into the image domain.

[0094] In some embodiments, the exponent of each probability in the probability set is determined, and the exponents are summed up; the sum is taken as the prediction result in the cross-entropy loss, and the exponent of the target updated probability is taken as the true value result, so as to determine the difference between the prediction result and the true value result, that is, the cross-entropy loss. In this way, taking the cross-entropy loss as the second loss for image domain division of the sample image set can reduce the difference between the predicted probability in the segmentation result and the domain label matched with the sample image.

[0095] In the embodiments of the present application, by normalizing the probability of the sample image belonging to each preset image domain, the image domain serving as the domain label of the sample image can be obtained, so that the updated probabilities in the updated probability set can be uniformly distributed, thereby enabling the domain labels to be uniformly distributed. Then, based on the target updated probability of the image domain to which each sample image belongs and the probability set, the cross-entropy loss is determined, and the cross-entropy loss is taken as the second loss for adjusting the sample image set division process, which can make the difference between the image domain to which the divided sample image belongs and the predicted probability smaller, thereby improving the accuracy of dividing the sample image set into multiple image domains.

[0096] In some embodiments, under the same preset misrecognition rate, a plurality of feature pairs are constructed by using the image feature set of the plurality of image domains and the preset feature queue, and the similarity between the features in each feature pair is analyzed, so as to determine the global threshold and the local threshold of each image domain, that is, the above step S102 can be implemented by the steps shown in Figure 2

[0097] In step S201, the second image feature set of the sample image in the at least two image domains and the preset feature queue are matched based on the preset misrecognition rate, to obtain a plurality of feature pairs.

[0098] In some embodiments, the preset feature queue is used to store the first image features of the sample image. After the first image features of the sample image are extracted, the first image features are copied into the preset feature queue.​

[0099] In some possible implementation manners, each feature pair includes a second image feature of a sample image and a feature in the preset feature queue. The feature pairs can include feature negative pairs and feature positive pairs; wherein the feature positive pairs represent that the second image feature of the sample image and the feature in the preset feature queue have high similarity, and the feature negative pairs represent that the second image feature of the sample image and the feature in the preset feature queue have low similarity.

[0100] The feature pairs are matched under the same preset false recognition rate in the plurality of image domains. After feature extraction and dimension conversion are performed on the sample images of the image domains, a second image feature set is obtained. Any second image feature is selected from the second image feature set, and any image feature is selected from the preset feature queue, similarity between the two features is analyzed, and whether the second image feature and the preset feature are feature negative pairs is determined according to the similarity and the preset false recognition rate.

[0101] In step S202, the global threshold and the local threshold corresponding to each image domain are determined based on the plurality of feature pairs.

[0102] In some embodiments, in the network training process, a plurality of feature negative pairs can be constructed by the above step S202, the global threshold can be set by the similarity in the plurality of feature negative pairs, and the local threshold can be set. In some possible implementation manners, the quantile of the similarity of the plurality of feature negative pairs is set as the global threshold, so that the global threshold can jointly affect each image domain. In the feature negative pairs included in each image domain, the similarity of the feature negative pairs in the image domain, the number of the feature negative pairs, and the preset false recognition rate can be obtained. In this way, the similarity of the feature negative pairs in the image domain is taken as the independent variable term of the exponential function, the unknown local threshold is taken as the threshold for judging whether the independent variable term is true or false, the number of the feature negative pairs is multiplied by the result of the exponential function, and the preset false recognition rate is taken as the multiplication result. In this way, the unknown local threshold can be inversely deduced by the preset false recognition rate as the multiplication result.

[0103] In the embodiments of the present application, under the same preset false recognition rate, a plurality of feature pairs are constructed from the second image feature set of the sample images and the preset feature queue. In this way, in the case that a sufficient number of features are stored in the preset feature queue, a sufficient number of feature pairs can be constructed. In this way, by a large number of feature pairs, a more accurate global threshold and a local threshold of each image domain can be set.

[0104] In some embodiments, a plurality of feature negative pairs are constructed by analyzing the similarity between the image features and the preset features in the queue, that is, the above step S201 can be implemented by the following steps S211 to S213 (not shown in the figure):

[0105] Step S211, determining the similarity between any second image feature in the second image feature set and any preset feature in the preset feature queue.

[0106] In some embodiments, the similarity between any optional second image feature in the second image feature set and any optional preset feature in the preset feature queue is analyzed. For each second image feature in the second image feature set, the similarity between the second image feature and each preset feature in the preset feature queue is analyzed; for example, if the preset feature queue includes 100 preset features, the similarity between each second image feature and the 100 preset features is analyzed.

[0107] Step S212, determining a similarity threshold value representing the similarity between the any image feature and the any preset feature based on the preset false recognition rate.

[0108] In some embodiments, the similarity threshold value required for the recognition result of the sample image in the plurality of image domains to reach the preset false recognition rate is analyzed according to the preset false recognition rate. In some possible implementations, when the preset false recognition rate is known, the number of feature pairs with recognition results of similarity required to reach the preset false recognition rate is determined according to the total number of feature pairs; in this way, by comparing the number of feature pairs with recognition results of similarity and the total number of feature pairs, the similarity threshold value required to reach the preset false recognition rate can be determined.

[0109] In one specific example, if the preset false recognition rate is (1e-4) and the total number of feature pairs is 100000, the top 10 feature pairs with recognition results of similarity are selected, i.e., the preset false recognition rate can be reached; and the similarity of the 10th feature pair can be set as the similarity threshold value.

[0110] Step S213, determining the any second image feature and the any preset feature with the similarity less than the similarity threshold value as a feature negative pair in the second image feature set and the preset feature queue to obtain the plurality of feature negative pairs.

[0111] In some embodiments, when the similarity is less than the similarity threshold value, it indicates that the second image feature and the preset feature in the feature pair are not similar, and then the second image feature and the preset feature are determined as a feature negative pair. In this way, by comparing the similarity of the plurality of feature pairs in the second image feature set and the preset feature queue, the plurality of feature negative pairs can be obtained.

[0112] In the steps S211 to S213, the similarity between each second image feature and each preset feature can be used to determine the probability that the second image feature and the preset feature belong to the same sample object, so as to determine whether the second image feature and the preset feature belong to a feature positive pair or a feature negative pair. Then, by analyzing whether the similarity meets the preset false recognition rate, the second image feature and the preset feature are regarded as a feature positive pair or a feature negative pair, so that the obtained multiple feature negative pairs can better match the preset false recognition rate, and the effectiveness of the multiple feature negative pairs in the network training process is improved.

[0113] In some embodiments, the global threshold of the multiple image domains and the local threshold of each image domain are set by analyzing the similarity of each feature negative pair, that is, the step S202 can be implemented by the following steps S221 and S222 (not shown in the figure):

[0114] In step S221, the global threshold is determined based on the similarity of the feature negative pairs in the multiple feature pairs.

[0115] In some embodiments, the similarity of the multiple feature negative pairs is obtained by obtaining the similarity of each feature negative pair in the multiple feature pairs in all image domains. Then, the global threshold is set by combining the similarities of the multiple feature negative pairs, so that the global threshold can cover each image domain. In some possible implementation manners, the quantile of the similarities of the multiple feature negative pairs is set as the global threshold, or the average value of the similarities of the multiple feature negative pairs is set as the global threshold, and the like.

[0116] In step S222, the local threshold corresponding to each image domain is determined based on the preset false recognition rate and the similarity of the feature negative pairs in each image domain.

[0117] In some embodiments, for any image domain, since the preset false recognition rate, the number of feature negative pairs, and the similarity of each feature negative pair are known, the unknown local threshold can be obtained by combining these known quantities with an exponential function in the formula for calculating the preset false recognition rate.

[0118] The steps S221 and S222 use the quantile of the similarities of the multiple feature negative pairs as the global threshold, which is accurate and convenient. The local threshold of each image domain under the preset false recognition rate can be obtained by the similarity of the feature negative pairs in one image domain, so that the local threshold corresponding to each image domain can be quickly obtained without increasing the amount of calculation.

[0119] In some embodiments, the image features of the extracted sample images are added into the feature queue to generate a preset feature queue for constructing feature pairs with the second image feature set, which can be implemented in the following two ways:

[0120] The first step is to obtain an initial feature queue for storing image features of sample images.

[0121] Here, the initial feature queue can be an empty queue or a queue that has stored image features of other sample images. The data amount of image features that the initial feature queue can store is set, and the number of image features of sample images that have been stored in the initial feature queue can be less than or equal to the set data amount. In this way, after feature extraction is performed on a sample image each time, a copy of the extracted image features is added to the initial feature queue.

[0122] The second step is to determine the available storage space of the initial feature queue when the number of image features stored in the initial feature queue is less than a preset feature threshold.

[0123] Here, the preset feature threshold is the number of features that the initial feature queue can store at most. If the number of image features stored in the initial feature queue is less than the preset feature threshold, it indicates that the initial feature queue is not full, and further indicates that the image features stored in the initial feature queue are not too old, because if the time is too long, the initial feature queue should have been full. In this case, the number of image features stored in the initial feature queue is counted, and the preset feature threshold is set, so that the number of image features that the initial feature queue can receive, i.e., the available storage space, is obtained.

[0124] The third step is to add the first image features in the first image feature set of the sample image to the initial feature queue to obtain the preset feature queue when the available storage space is greater than or equal to the storage space required by the first image feature set.

[0125] Here, if the available storage space is greater than or equal to the storage space required by the first image feature set of the sample image, it indicates that the available storage space is sufficient to accommodate the first image feature set of the sample image, i.e., the initial feature queue can still store the first image features in the first image feature set. In this case, the image features of the extracted sample image can be directly added to the initial feature queue to obtain the preset feature queue. In this way, when the available storage space in the initial feature queue is sufficient, the image features of the extracted sample image can be directly added to the initial feature queue to obtain the preset feature queue, so that the features in the preset feature queue have a certain similarity with the image features of the sample image extracted in real time, and the queue for storing image features of sample images can be quickly constructed.

[0126] In the second mode, in a first step, when the number of image features stored in the initial feature queue reaches the preset number of features, or when the number of image features stored in the initial feature queue is less than the preset number of features and the available storage space of the initial feature queue is less than the number of features in the first image feature set of the sample image, the first image feature in the first image feature set is added to the initial feature queue to obtain a to-be-adjusted queue.

[0127] Here, if the number of image features stored in the initial feature queue reaches the preset number of features, it means that the initial feature queue is full and cannot continue to store newly extracted image features. Alternatively, if the number of image features stored in the initial feature queue does not reach the preset number of features, but the available storage space is less than the number of features in the first image feature set, it means that after the first image feature set is stored in the initial feature queue, the storage capacity of the queue exceeds the set maximum storage capacity, i.e., exceeds the preset number of features. In this case, the first image feature is added to the initial feature queue first, and then the next step of outputting the old image features is performed.

[0128] In a second step, according to the storage order of the to-be-adjusted queue, the initial features matching the number of features in the first image feature set are outputted to obtain the preset feature queue.

[0129] Here, according to the number of features in the first image feature set and the available storage space of the initial feature queue (if the number of image features stored in the initial feature queue reaches the preset number of features, the available storage space is 0), the number of initial features to be outputted is determined so that the queue after outputting the initial features has enough storage space to store the first image features. While adding the first image feature to the initial feature queue, according to the storage order in the queue, i.e., the first-in-first-out order, a part of the initial features stored first in the to-be-adjusted queue are outputted, and the obtained preset feature queue is a queue with a number of features equal to the preset number of features. In this way, in the case where there is not enough space in the initial feature queue to store the first image features in the first image feature set, the first image features are added to the initial feature queue while the first-in-first-out feature of the queue is outputted in the to-be-adjusted queue, so that the queue for storing image features can be updated in time.

[0130] In some embodiments, for each image domain, the difference information between the local threshold and the global threshold of the image domain is analyzed, and the difference information is used to dynamically adjust the classification loss of the image domain, so that the first loss can adaptively change with the local threshold of the image domain, i.e., the above step S103 can be realized by the following steps S131 to S133 (not shown in the figure):

[0131] Step S131, determining difference information between the local threshold of each image domain and the global threshold.

[0132] In some embodiments, the difference information represents the numerical difference between the local threshold and the global threshold of each image domain. For any image domain, the difference information can be the ratio of the local threshold to the global threshold of the image domain.

[0133] Step S132, assigning the difference information to the classification loss of each image domain in the object recognition network to be trained to obtain a weighted classification loss.

[0134] In some embodiments, the difference information is multiplied by the classification loss as a weight of the classification loss to obtain a weighted classification loss.

[0135] Step S133, fusing the weighted classification losses corresponding to the at least two image domains to obtain the first loss.

[0136] In some embodiments, the weighted classification loss of each image domain is determined, and the weighted classification losses of all image domains are element-wise summed to realize fusion of multiple weighted classification losses to obtain the first loss.

[0137] The steps S131 to S133 above use the difference information between the local threshold of each image domain and the global threshold as the weight of the sample image in the image domain, improve the consistency between different image domains, and thus realize dynamic adjustment of the classification loss of the image domain using the difference information. Thus, the multiple classification losses after dynamic adjustment are fused as the first loss of the training network, so that the trained network has good fairness, i.e., the trained network can balance images in different domains.

[0138] In some embodiments, the training process of the object network to be trained is realized by fusing the second loss and the first loss, i.e., the above step S104 can be realized by the following steps S141 to S143 (not shown in the figure):

[0139] Step S141, assigning a preset weight value to the second loss to obtain an adjusted second loss.

[0140] Here, the preset weight is the weight of the second loss in the image domain division process, which can be set based on experience, for example, the preset weight is set to 3. The preset weight is multiplied by the second loss to obtain the adjusted second loss.

[0141] Step S142, fusing the adjusted second loss and the first loss to obtain a total loss.

[0142] Here, the adjusted second loss and the first loss are element-wise summed to obtain a total loss.

[0143] At step S143, the network parameters of the to-be-trained object recognition network are adjusted based on the total loss, so that the loss output by the adjusted object recognition network satisfies the preset convergence condition.

[0144] Here, in the iteration process, the network parameters in the to-be-trained object recognition network are adjusted based on the total loss until the loss of the to-be-trained object recognition network satisfies the convergence condition, the iteration process is stopped, the training of the to-be-trained object recognition network is completed, and the adjusted object recognition network is obtained. The network parameters at least include one of the following: weights, learning rates, and the like.

[0145] Embodiments of the present application provide an object recognition method, as shown in Figure 3 as shown in Figure 3 as shown in

[0146] At step S301, an image to be recognized including an object to be recognized is obtained.

[0147] Here, in the object recognition network, the image to be recognized can be an image including an object to be recognized in a picture, and the image to be recognized includes a preset restriction condition of the object to be recognized. The object to be recognized can be a person, an animal or other object to be recognized at any age. Taking the object to be recognized as a person as an example, the image to be recognized can include facial features or body features, and the like, and the facial features or body features are matched with images satisfying the conditions in a preset image library. In some possible implementation manners, the image to be recognized including any person can be a video frame in a video collected by a camera in a traffic scene; or can be a video frame in a video collected by a camera in a campus scene; or can be a video frame in a video collected by a camera in a railway station scene, and the like.

[0148] At step S302, the image to be recognized is feature-extracted based on the object recognition network to obtain image features.

[0149] In some embodiments, the object recognition network is trained based on the method described in the above embodiments. The object recognition network can be trained by the training method of the object recognition network provided in the above embodiments, and can be used for feature extraction, feature classification, and determining the matched multiple frames of images according to the similarity of the extracted image features. The feature extraction function in the object recognition network to be trained can be implemented by using a convolution layer, and the object recognition function can be implemented by using a pooling layer and / or a fully connected layer. The image to be recognized including the object to be recognized is input into the object recognition network for feature extraction to obtain the image features of the object to be recognized.

[0150] In step S303, based on the image features, the target preset image in which the marked object in the picture matches the object to be recognized is searched in the preset image library.

[0151] In some embodiments, the preset image library is used to store multiple frames of preset images in which the picture includes the marked object. The preset image library includes multiple frames of preset images in which the identity information of the marked object is labeled, and the picture of the multiple frames of preset images can include objects in different image domains; for example, the objects in different image domains in the preset images have different origins, different visual features, or belong to different organizations, etc.

[0152] In the multiple frames of preset images in the preset image library, the image domain of the preset object corresponding image can be the same as or different from the image domain of the image to be recognized. The preset image library can be a static image library including a large number of images, and each frame of image corresponds to the identity information of the object labeled. The identity information can be information used to identify the identity of the object, including but not limited to at least one of the following: name, age, gender, hometown, ID number, height, and blood type. The image features of the image to be recognized are used to search for the matched multiple frames of target preset images in the preset image library, and the similarity between the preset object in the multiple frames of target preset images matched in the preset image library and the object to be recognized is high.

[0153] In one specific example, the object to be recognized and the preset objects including multiple different domains can be respectively input into the object recognition network, the image features are extracted, the image matching is performed according to the similarity between the image features of the object to be recognized and the preset objects, and the most similar preset object to the object to be recognized is determined in the preset image library.

[0154] In some embodiments, the preset image library can be divided into multiple groups of preset images according to the different domains of the preset objects in the images, so that the most similar preset object to the object to be recognized can be matched in multiple different image domains.

[0155] In the embodiments of this application, in the scenario of cross-image domain object recognition, the accuracy of cross-image domain object recognition can be improved by finding matching images through image features extracted by the adjusted object recognition network, thereby improving the accuracy of determining the identity information of the object to be identified based on the identity information of the matching object.

[0156] The following will illustrate an exemplary application of the embodiments of this application in a real-world application scenario, using the example of improving the performance of a face recognition network by adaptively assigning different weights to samples.

[0157] Deep learning-based face recognition (FR) models calculate the correct acceptance rate at a predefined false acceptance rate (e.g., (1e-4)). Figure 4 As shown, Figure 4 A schematic diagram illustrating the similarity distribution of two datasets provided in a related embodiment, such as... Figure 4 As shown in (a), the similarity distribution of two datasets under the same false recognition rate is obtained by... Figure 4 (a) Visualizes the distribution of similarity scores from two datasets and observes that dataset 1 and dataset 2 have different thresholds for the same false recognition rate, i.e., different thresholds on different datasets. Figure 4 In the diagram, curves 401 and 402 represent the positive and negative pair similarities in dataset 1, respectively; curves 403 and 404 represent the positive and negative pair similarities in dataset 2, respectively; as shown... Figure 4 As shown in (a), there is a significant difference between threshold 1 and threshold 2 under the same FAR. Furthermore, in Figure 4 As shown in (b), among related technologies, the FR model performs well on both datasets, but... Figure 4 In (b) and (c) of the data, the performance results of the FR model are highly sensitive to changes in the threshold. For example... Figure 4 As shown in (b), when using a threshold of 1 from dataset 1 to evaluate the performance of dataset 2, a large number of false negative (FN) samples are generated; as Figure 5 As shown in (c), when using threshold 2 of dataset 2 to evaluate the performance of dataset 1, a large number of false positive (FP) samples are generated. Furthermore, when the FR model is applied to industrial applications, only a single fixed threshold is used for all scenarios. This indicates that the current method of setting different thresholds for different datasets is not fully compatible with actual FR practices. In other words, the method of setting different thresholds for different datasets in related technologies does not conform to practical applications and cannot effectively improve the performance of object recognition networks.

[0158] Based on this, the embodiment of the present application provides a face recognition method based on a One-Threshold-for-All (OTA) evaluation protocol to better utilize the overall performance and fairness of the threshold of the FR model on multiple data sets, and introduces an effective threshold consistency penalty (TCP) scheme to solve the threshold imbalance problem in the training process.

[0159] In the OTA evaluation protocol, the embodiment of the present application uses a fixed threshold to measure the performance of different data sets, which is more consistent with the actual FR, and can be easily combined with the existing evaluation protocol. The fixed threshold deployed is referred to as the global threshold t c . Given (G) data sets and overall false rejection rate, the embodiment of the present application uses an additional data set referred to as a calibration data set to propose a global threshold estimation method, in which the negative logarithm in the calibration data set is relatively acceptable. In addition, the calibration data set should cover as many domain images as possible. After preparing the calibration data set, the threshold of the data set at the overall false rejection rate can be used as t c . Under the OTA evaluation protocol, the correct acceptance rate and false rejection rate results of the multiple G data sets are calculated based on t c respectively. At the same time, the embodiment of the present application proposes a new fairness index γ to represent the threshold imbalance degree of the multiple G data sets. The threshold wherein, in the entire false rejection rate is the local threshold of the gth data set in the multiple G data sets. In this way, the fairness index γ can be obtained according to the global threshold t c and

[0160] ​In the embodiments of the present application, the threshold imbalance between different domains in training is mitigated by a threshold consistency penalty to improve the performance under the OTA evaluation protocol. In some possible implementation ways, first, since the domain labels of the training dataset are usually unavailable, and the domain labels need to be obtained to calculate the thresholds of different domains, the embodiments of the present application implicitly assign domain labels to samples by using an implicit domain division (IDD) module to divide each mini-batch of samples into M domains. And the calibration value and the thresholds of different domains are calculated by constructing negative pairs. In order to generate accurate thresholds, the embodiments of the present application establish a feature queue to maintain the features of previous iterations. During the training process, first, negative pairs can be constructed according to the features of the current batch and the features of the feature queue; then, the similarity of these negative pairs is determined to obtain a global threshold. In addition, the embodiments of the present application use the features of the mth domain of the current batch and the features of the feature queue to generate the local threshold of the mth domain, wherein the domain labels of each mini-batch of sample image sets are predicted by the IDD module. Finally, the ratio of the local threshold and the global threshold is used as the loss weight of the sample of the mth domain to punish the samples with high local threshold, aiming to reduce the degree of threshold imbalance. In one specific example, for the training dataset of collected face images, the embodiments of the present application first implicitly assign domain labels to the training dataset of face images, thereby dividing the training dataset of face images into M image domains; then, the global threshold and the local thresholds of different domains are determined by constructing negative pairs. In this way, during the training process of the face recognition network, the loss weight of each image domain is dynamically adjusted by the ratio of the local threshold and the global threshold of the face image, which can improve the consistency between the local thresholds of different image domains, and thus can improve the recognition accuracy of the face recognition network for different face images.

[0161] The embodiments of the present application use the 1:1 face verification evaluation protocol to demonstrate the evaluation process of the FR model. Both FAR and TAR are used for face verification. Given N p positive pairs, the expression of TAR is shown in formula (1):

[0162]

[0163] where t is the selected similarity score threshold, is the similarity score of the jth positive pair. I(x) is an indicator function that returns 1 when x is true and 0 when x is false. Similarly, given N n negative pairs, the expression of FAR is shown in formula (2):

[0164]

[0165] where, is the similarity score of the ith negative pair.

[0166] During the test, a value of FAR is usually fixed (e.g., FAR is set to (1e-4)); and the corresponding value of TAR is determined to represent the performance of the FR model. For each dataset, the threshold t at a specific β in formula (2) can be generated by the quantile of the similarity scores of all negative pairs. Based on the similarity scores of all positive pairs and the threshold t, the α in formula (1) can be determined.

[0167] In the embodiments of the present application, the object recognition network includes a common OTA evaluation protocol and a threshold consistency penalty (TCP) scheme.

[0168] Although the above-mentioned OTA evaluation protocol has been applied in many scenarios, the current evaluation protocol is not compatible with the actual FR model, because even if the FR model and the pre-defined FAR are the same, different threshold values are used for different test datasets. In addition, it can be observed that the threshold imbalance phenomenon, when these datasets come from different domains, the threshold values of these datasets vary greatly. In contrast, when the FR model is actually deployed, only one fixed threshold value (i.e., global threshold) is applied to all scenarios, which means that the current evaluation protocol is not completely consistent with the actual FR model application. Therefore, it is crucial to use a fixed FR threshold to evaluate the performance of all domains.

[0169] The evaluation process of the OTA on the G test dataset is described in the embodiments of the present application. Given the value of the global FAR (e.g., (1e-4)), first, a fixed global threshold is generated. Then, based on the global threshold, the values of TAR and FAR of different datasets are determined. Finally, a fairness index is defined to represent the degree of imbalance between the thresholds of these datasets.

[0170] In OTA, given the G test dataset, the global threshold estimation can be achieved in two ways. In the first way, first, multiple G datasets are combined into a complete dataset, and the features of all negative pairs are extracted to determine the similarity of the negative pairs. Then, the threshold score at the global FAR is obtained as the global threshold t c The first way is to estimate the threshold at the overall FAR based on the negative pairs constructed by all datasets, which is feasible and effective when the number of negative pairs is relatively acceptable. However, when the number of negative pairs increases, the threshold estimation requires a large amount of computational cost. Therefore, another way is proposed in the embodiments of the present application, which realizes the global threshold estimation by using an additional dataset (referred to as a calibration dataset). It is assumed that the calibration dataset covers as many domain images as possible, so that the global threshold t cIt is general and accurate. Furthermore, the size of the calibration dataset is acceptable, making the computational cost affordable. Similarly, by determining the similarity of all negative pairs from the calibration dataset, a global threshold t under overall FAR is obtained. c .

[0171] Obtaining the global threshold t c Then, for the g-th dataset, where g∈{1,…,G}, it is possible to achieve the threshold t c Generate the α of TAR in the g-th dataset. g and FAR's β g The mean of all datasets is used. and variance The overall TAR performance of these datasets is represented by the logarithmic value. Since FAR values ​​are typically small, and the FAR values ​​vary significantly across different domains at the same threshold, a logarithmic value is used. 10 This operation is used to maintain monotonicity and simplify calculations. Furthermore, without this operation, the statistics would be dominated by large FAR values. For example, the mean of {1e-3, 1e-4, 1e-5} is predominantly (1e-3), where (1e-5) is ignored. Therefore, for the values ​​of FAR, the mean of all datasets is used. and variance Furthermore, embodiments of this application propose a fairness metric, denoted as γ, to represent the degree of imbalance among thresholds in dataset G. Embodiments of this application also determine the threshold within the g-th dataset under the global FAR value. Will This is called the local threshold for the g-th dataset. Simultaneously, the local threshold G (i.e., ) and global threshold t c The deviation between them is defined as the fairness index γ, which can be expressed as formula (3):

[0172]

[0173] In formula (3), the more severe the imbalance between the thresholds of different domains, the larger γ becomes.

[0174] In this embodiment of the application, the TCP implementation process is as follows: Figure 5 As shown, Figure 5 This diagram illustrates the TCP implementation framework provided in an embodiment of this application, which includes IDD 501 and Calibration and Domain Thresholds Estimation (CDTE) 502. In each training iteration, firstly, the FR model 52 is used to extract features from each mini-batch of images 51. 53and update the feature queue Q 54, where N is the number of samples in each mini-batch image. Then, the IDD 501 is used to Implicitly partitioning into M domains 55, here, the domain loss L d is determined in the CDTE 502. c 56and local thresholds for each image domain, such as, includes Finally, the loss L c of the TCP is determined based on t and the real identity label . t .

[0175] Thus, an implementation framework of the TCP is described in Figure 5 to reduce the threshold imbalance between different domains in training, where a domain is defined as a set of samples that share common visual or non-visual attributes of the FR model. In the IDD 501, each mini-batch image is partitioned into several domains without requiring additional annotations. In the CDTE 502, first, enough negative pairs are constructed using the features of the current batch and the features in the extracted feature queue. Then, the similarity of these negative pairs is determined and the global threshold and local thresholds are estimated. Finally, based on the global threshold and local thresholds, the loss weight of samples from each domain is adaptively adjusted.

[0176] Since the domain labels in the training dataset are usually unavailable, the IDD is trained in a self-supervised manner without any explicit real information to predict the domain label of each sample in the embodiments of the present application. As shown in Figure 6 , the IDD 501 includes two fully connected layers and an activation function (softmax) layer implementation to take the feature representation f i of the i-th sample as input and predict the domain probability as shown in equation (4):

[0177]

[0178] where H denotes the neural network of the IDD. M is a predefined hyperparameter about the number of implicit domains, which is irrelevant to the number of evaluation datasets (i.e., the G dataset in the OTA). is the domain probability of the m-th domain. In addition, H is trained by a self-distribution label strategy. is the initial predicted domain probability of the i-th sample. To generate uniformly distributed domain labels, the embodiments of the present application use a revised probability adjusted by a prior probability, the updated domain probability of the m-th domain as shown in equation (5):

[0179]

[0180] where T is the number of samples to compute the expectation. T is set to the number of samples in each mini-batch image. Thus, the expectation of the normalized probability is Domain label d of the i-th sample i ∈ {1, …, M} is Meanwhile, to reduce the difference between the predicted domain probability and the generated domain label, the domain loss L d As shown in formula (6):

[0181]

[0182] where N is the number of samples in each iteration.

[0183] In the embodiment of the present application, the implementation effect of dividing the domain in the IDD is as shown in Figure 6 Figure 7 The visualization result diagram of dividing the domain in the IDD provided by the embodiment of the present application, wherein the result 601 represents the distribution of the similarity scores of different persons in the personnel image set 1, the vertical line 61 represents the local threshold value in the personnel image set 1 under the FAR with the false recognition rate value of (1e-4); the result 602 represents the distribution of the similarity scores of different persons in the personnel image set 2, and the vertical line 62 represents the local threshold value in the personnel image set 2 under the FAR with the false recognition rate value of (1e-4). From the corresponding values of the vertical lines 61 and 62, the difference between the two threshold values is small, which indicates that the implicit grouping provided by the embodiment of the present application can alleviate the threshold imbalance between different domains.

[0184] In the embodiment of the present application, the implementation framework of the TCP adjusts the imbalance degree between the threshold values of different domains, Figure 8 The visualization result diagram of the threshold fairness of different methods in the training process provided by the embodiment of the present application, wherein the curve 701 represents the fairness value curve between the threshold values of different domains obtained by training without the TCP framework penalty term; the curve 702 represents the fairness value curve between the threshold values of different domains obtained by training with the TCP framework penalty term. From the curve 701 and the curve 702, it can be seen that the embodiment of the present application is very effective for alleviating the threshold imbalance between different domains by performing multiple iteration training with the TCP framework penalty term.

[0185] By constructing a sufficient number of high-quality negative pairs, the global threshold value and the local threshold value in the training process are estimated. For unsupervised learning, the memory bank of the previous mini-batch is used to obtain sufficient negative samples. In some possible implementation manners, by constructing a feature queue ​To construct a sufficient number of negative pairs, K is the number of iterations, and d represents the dimension of the feature representation extracted by the neural network for each face image. Meanwhile, the feature drift of the FR model is slow, meaning that previously extracted features can be considered approximations of the current network's output within a certain number of training steps. Therefore, K can be set to a relatively large value (e.g., K set to 1000) to generate a sufficient number of high-quality negative pairs. Furthermore, an auxiliary label queue is established. This is used to store the identity labels of the features in Q. In each iteration, the features of the current batch are extracted first. Where y i It is the label corresponding to f. Then, the features. and tags Features and labels are respectively placed into feature queue Q and label queue Q′, with the first batch of features and labels in Q and Q′ being dequeued. Finally, based on... Construct a negative pair with Q.

[0186] For global threshold estimation, the similarity of all negative pairs is determined, and a global threshold t is generated during training under the overall FAR value. c For local threshold estimation, domain labels are first generated for the current mini-batch of samples based on IDD. Then, to accurately estimate the threshold distribution for each domain m, features of samples with the same domain label (i.e., m) are selected to construct domain-specific negative pairs with features from feature queue Q. By determining the similarity of such domains and the specific negative pairs, the local threshold for the m-th domain at the same FAR value can be obtained. Finally, the calibration threshold t is obtained. c and local threshold

[0187] After generating calibration and local thresholds, the TCP loss L is defined at the domain level. t As shown in formula (7):

[0188]

[0189] Among them, T m It is an index set containing the indices of samples with domain label m in each mini-batch. N is the number of samples in each mini-batch, and L is the index of the sample with domain label m. i This represents the classification loss for the i-th sample. The TCP loss forces the neural network to pay more attention to the data from... Hard-domain samples, and automatically reduce these from during training. The contribution of samples from each domain. In other words, the TCP loss aims to reduce the threshold imbalance across multiple domains by dynamically adjusting the loss weights of the M domains. Based on this, the overall loss function is shown in Equation (8):

[0190] L = L t +λLd (8);

[0191] wherein, λ is the loss weight of the domain loss L d in the IDD.

[0192] In the embodiments of the present application, the limitations of the FR evaluation protocol in the related art are studied, and an OTA evaluation protocol more consistent with the actual application stage is proposed. In addition, a threshold consistency penalty scheme is also proposed to improve the performance and fairness of the object recognition model under the OTA protocol.

[0193] The embodiments of the present application provide a training device of an object recognition network, Figure 8 The structure of the training device of the object recognition network in the embodiments of the present application is shown in FIG. 8, which comprises: Figure 9

[0194] The first acquisition module 801 is configured to acquire at least two image domains including a sample object.

[0195] The first determination module 802 is configured to determine, based on a preset misrecognition rate, a global threshold for evaluating the similarity of the sample object in the at least two image domains and a local threshold for evaluating the similarity of the sample object in each image domain.

[0196] The first adjustment module 803 is configured to adjust the classification loss of each image domain in the to-be-trained object recognition network based on the global threshold and the local threshold of each image domain, to obtain a first loss.

[0197] The second adjustment module 804 is configured to adjust the network parameters of the to-be-trained object recognition network based on the first loss and a second loss of the at least two image domains, so that the loss output by the adjusted object recognition network satisfies a preset convergence condition.

[0198] In some embodiments, the first acquisition module 801 comprises: a first acquisition sub-module configured to acquire a sample image set in which the attribute information of the sample object has been labeled; and a first segmentation sub-module configured to segment, in the to-be-trained object recognition network, the sample image set into the at least two image domains based on the attribute information of the sample object and a plurality of preset image domains of the labeled domain labels.

[0199] ​In some embodiments, the first segmentation sub-module comprises: a first extraction unit configured to perform feature extraction on the set of sample images in the object recognition network to be trained to obtain a first set of image features; a first transformation unit configured to transform the feature dimensions in the first set of image features to obtain a second set of image features; and a first determination unit configured to determine, based on the attribute information of the second image features in the second set of image features, an image domain to which the second image features match in the plurality of preset image domains.

[0200] In some embodiments, the first determination unit comprises: a first prediction sub-unit configured to predict, based on the attribute information of the second image features, a probability of a domain label of each preset image domain to which the second image features match; and a first determination sub-unit configured to determine, based on the probability, an image domain to which a sample image belonging to the second image features matches.

[0201] In some embodiments, the apparatus further comprises: a first normalization module configured to normalize a probability of a sample image belonging to each preset image domain in the set of sample images to obtain an updated probability set; a second determination module configured to determine, in the updated probability set, a target updated probability that satisfies a preset condition; and a third determination module configured to determine, as the second loss of segmenting the sample image to an image domain, a cross-entropy loss between the target updated probability and the probability set.

[0202] In some embodiments, the first determination module 802 comprises: a first matching sub-module configured to match, based on the preset misrecognition rate, a second set of image features of a sample image in the at least two image domains and a preset feature queue to obtain a plurality of feature pairs; and a first determination sub-module configured to determine, based on the plurality of feature pairs, the global threshold value and the local threshold value corresponding to each image domain.

[0203] In some embodiments, the plurality of feature pairs comprises a plurality of feature negative pairs, and the first matching sub-module comprises: a second determination unit configured to determine a similarity between any second image feature in the second set of image features and any preset feature in the preset feature queue; a third determination unit configured to determine, based on the preset misrecognition rate, a similarity threshold value representing the similarity between the any image feature and the any preset feature; and a fourth determination unit configured to determine, in the second set of image features and the preset feature queue, the any second image feature and the any preset feature as a feature negative pair if the similarity is less than the similarity threshold value, to obtain the plurality of feature negative pairs.

[0204] In some embodiments, the first determining sub-module comprises: a fifth determining unit configured to determine the global threshold based on the similarity of the feature-negative pairs in the plurality of feature pairs; and a sixth determining unit configured to determine the local threshold corresponding to each image domain based on the preset false recognition rate and the similarity of the feature-negative pairs in each image domain.

[0205] In some embodiments, the apparatus further comprises: a second obtaining module configured to obtain an initial feature queue for storing image features of sample images; a fourth determining module configured to determine available storage space of the initial feature queue in a case where the number of image features stored in the initial feature queue is less than a preset feature number; and a first adding module configured to add first image features in a first image feature set of the sample image to the initial feature queue to obtain the preset feature queue in a case where the available storage space is greater than or equal to storage space required by the first image feature set.

[0206] In some embodiments, the apparatus further comprises: a second adding module configured to add first image features in a first image feature set of the sample image to the initial feature queue to obtain a to-be-adjusted queue in a case where the number of image features stored in the initial feature queue reaches the preset feature number, or the number of image features stored in the initial feature queue is less than the preset feature number and the available storage space of the initial feature queue is less than the feature number of the first image feature set; and a first output module configured to output initial features in the to-be-adjusted queue that match the feature number of the first image feature set in a storage order of the to-be-adjusted queue to obtain the preset feature queue.

[0207] In some embodiments, the first adjusting module 803 comprises: a second determining sub-module configured to determine difference information between the local threshold of each image domain and the global threshold; a first assigning sub-module configured to assign the difference information to a classification loss of the to-be-trained object recognition network in each image domain to obtain a weighted classification loss; and a first fusing sub-module configured to fuse the weighted classification losses corresponding to the at least two image domains to obtain the first loss.

[0208] In some embodiments, the second adjusting module 804 comprises: a second assigning sub-module configured to assign a preset weight value to the second loss to obtain an adjusted second loss; a second fusing sub-module configured to fuse the adjusted second loss and the first loss to obtain a total loss; and a first adjusting sub-module configured to adjust network parameters of the to-be-trained object recognition network based on the total loss, so that the loss output by the adjusted object recognition network satisfies the preset convergence condition.

[0209] Embodiments of the present application provide an object recognition apparatus,Figure 9 An object recognition device is shown in FIG. 1. As shown in FIG. 1, the object recognition device 900 includes: Figure 10

[0210] A third obtaining module 901 is configured to obtain a to-be-recognized image including a to-be-recognized object.

[0211] A first extracting module 902 is configured to perform feature extraction on the to-be-recognized image based on an object recognition network to obtain image features, wherein the object recognition network is trained based on the method of the first aspect.

[0212] A first searching module 903 is configured to search, based on the image features, a target preset image in which a marked object in a picture matches the to-be-recognized object from a preset image library.

[0213] It should be noted that the above description of the device embodiments is similar to the description of the method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the device embodiments of the present application, please refer to the description of the method embodiments for understanding. It should be noted that, in the embodiments of the present application, if the training method of the object recognition network is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a terminal, a server, etc.) to execute all or part of the method described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a hard disk drive, a read-only memory (ROM), a magnetic disk or an optical disk, and various other storage media that can store program codes. Thus, the embodiments of the present application are not limited to any specific hardware and software combination.

[0214] The embodiments of the present application further provide a computer program product, which includes computer executable instructions. After the computer executable instructions are executed, the training method of the object recognition network or the object recognition method provided by the embodiments of the present application can be realized.

[0215] The embodiments of the present application further provide a computer storage medium, which stores computer executable instructions. When the computer executable instructions are executed by a processor, the training method of the object recognition network or the object recognition method provided by the above embodiments can be realized.

[0216] The embodiments of the present application provide a computer device, Figure 10 ​A schematic diagram of a computer device according to an embodiment of the present application is shown in ​ As shown in the figure, the computer device 1000 includes a processor 1001, at least one communication bus, a communication interface 1002, at least one external communication interface, and a memory 1003. The communication interface 1002 is configured to realize the connection and communication between the components. The communication interface 1002 can include a display screen, and the external communication interface can include a standard wired interface and a wireless interface. The processor 1001 is configured to execute an image processing program in the memory to implement the training method of the object recognition network or the object recognition method provided in the above embodiments.

[0217] The above description of the training device of the object recognition network, the computer device, and the storage medium is similar to the description of the method embodiments, has similar technical effects and advantages as the method embodiments, and is limited in length. The above description of the method embodiments is not repeated here. For technical details not disclosed in the training device of the object recognition network, the computer device, and the storage medium embodiments of the present application, please refer to the description of the method embodiments.

[0218] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures, or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of each process does not mean the execution order, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The sequence number of the above embodiments of the present application is only for description, not representing the advantages and disadvantages of the embodiments. It should be noted that in this paper, the term "include", "contain" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0219] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other manners. The described device embodiments are merely illustrative. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling between the components, or the direct coupling or communication connection between the components, can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0220] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units; they can be located in one place, or distributed on multiple network units; and some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0221] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be separately taken as one unit, or two or more units can be integrated in one unit; the integrated unit can be realized in the form of hardware or in the form of hardware plus software function unit. Those skilled in the art can understand that all or part of the steps of the foregoing method embodiments can be completed by a program instruction related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program is executed to perform the steps of the foregoing method embodiments; and the foregoing storage medium includes a mobile storage device, a read-only memory (ROM), a magnetic disc or an optical disc, and various media that can store program codes. Alternatively, when the foregoing integrated unit is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in the embodiments of the present application. The foregoing storage medium includes a mobile storage device, a ROM, a magnetic disc or an optical disc, and various media that can store program codes. The foregoing is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any skilled person in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for training an object recognition network, the method comprising: The method comprises: Obtaining a sample image set of attribute information of a labeled sample object; in a to-be-trained object recognition network, based on the attribute information of the sample object and a plurality of preset image domains of a labeled domain label, the sample image set is divided into at least two image domains; the probability that a sample image in the sample image set belongs to each preset image domain is normalized to obtain an updated probability set; in the updated probability set, a target updated probability that meets a preset condition is determined; the cross-entropy loss between the target updated probability and the probability set is determined as a second loss of dividing the sample image into an image domain; based on a preset misrecognition rate, a global threshold for evaluating the similarity of the sample object in the at least two image domains and a local threshold for evaluating the similarity of the sample object in each image domain are determined; based on the global threshold and the local threshold of each image domain, the classification loss of each image domain in the to-be-trained object recognition network is adjusted to obtain a first loss; based on the first loss and the second loss of the at least two image domains, the network parameters of the to-be-trained object recognition network are adjusted, so that the loss output by the adjusted object recognition network meets a preset convergence condition.

2. The method of claim 1, wherein, The method comprises: In the to-be-trained object recognition network, feature extraction is performed on the sample image set to obtain a first image feature set; the feature dimension in the first image feature set is transformed to obtain a second image feature set; based on the attribute information of the second image feature in the second image feature set, the image domain matched by the second image feature is determined in the plurality of preset image domains.

3. The method of claim 2, wherein, The method comprises: Based on the attribute information of the second image feature, the probability that the second image feature matches the domain label of each preset image domain is predicted; based on the probability, the image domain matched by the sample image to which the second image feature belongs is determined.

4. The method according to any one of claims 1 to 3, characterized in that, The method comprises: Based on the preset misrecognition rate, the second image feature set of the sample image in the at least two image domains and the preset feature queue are matched to obtain a plurality of feature pairs; Based on the plurality of feature pairs, the global threshold and the local threshold corresponding to each image domain are determined.

5. The method of claim 4, wherein, The plurality of feature pairs comprise a plurality of feature negative pairs, and the method comprises: Based on the preset misrecognition rate, the second image feature set of the sample image in the at least two image domains and the preset feature queue are matched to obtain a plurality of feature pairs, which comprises: determine a similarity between any second image feature in the second image feature set and any preset feature in the preset feature queue; determine a similarity threshold value representing the similarity between any second image feature and the any preset feature based on the preset misrecognition rate; determine the any second image feature and the any preset feature as a feature negative pair if the similarity between the any second image feature and the any preset feature is less than the similarity threshold value, in the second image feature set and the preset feature queue, to obtain the plurality of feature negative pairs.

6. The method of claim 4, wherein, The determining the global threshold value and the local threshold value corresponding to each image domain based on the plurality of feature pairs comprises: determining the global threshold value based on the similarity of the feature negative pairs in the plurality of feature pairs; and determining the local threshold value corresponding to each image domain based on the preset misrecognition rate and the similarity of the feature negative pairs in each image domain.

7. The method of claim 4, wherein, Before the matching the second image feature set of the sample images and the preset feature queue in the at least two image domains based on the preset misrecognition rate to obtain the plurality of feature pairs, the method further comprises: obtaining an initial feature queue for storing image features of sample images; determining available storage space of the initial feature queue in a case that the number of image features stored in the initial feature queue is less than a preset feature number; and adding first image features in a first image feature set of the sample images to the initial feature queue to obtain the preset feature queue in a case that the available storage space is greater than or equal to storage space required by the first image feature set.

8. The method of claim 4, wherein, Before the matching the image feature set and the preset feature queue based on the preset misrecognition rate to obtain the plurality of feature pairs, the method further comprises: in a case that the number of image features stored in the initial feature queue reaches the preset feature number, or the number of image features stored in the initial feature queue is less than the preset feature number and the available storage space of the initial feature queue is less than the feature number of the first image feature set of the sample images, adding first image features in the first image feature set to the initial feature queue to obtain a to-be-adjusted queue; and outputting initial features in the to-be-adjusted queue that match the feature number of the first image feature set in a storage order of the to-be-adjusted queue to obtain the preset feature queue.

9. The method according to any one of claims 1 to 3, characterized in that, The adjusting the classification loss of each image domain in the to-be-trained object recognition network based on the global threshold value and the local threshold value of each image domain to obtain a first loss comprises: determining difference information between the local threshold value of each image domain and the global threshold value; assigning the difference information to the classification loss of each image domain in the to-be-trained object recognition network to obtain a weighted classification loss; and fusing the weighted classification losses corresponding to the at least two image domains to obtain the first loss.

10. The method according to any one of claims 1 to 3, characterized in that, The adjusting the network parameters of the to-be-trained object recognition network based on the first loss and the second loss of the at least two image domains, so that the loss output by the adjusted object recognition network satisfies a preset convergence condition, comprises: assigning a preset weight value to the second loss to obtain an adjusted second loss; fusing the adjusted second loss and the first loss to obtain a total loss; and adjusting the network parameters of the to-be-trained object recognition network based on the total loss, so that the loss output by the adjusted object recognition network satisfies the preset convergence condition.

11. An object recognition method characterized by, The method comprises: obtaining a to-be-recognized image including a to-be-recognized object; performing feature extraction on the to-be-recognized image based on an object recognition network to obtain image features; wherein the object recognition network is trained based on the method of any one of claims 1 to 10; and searching for a target preset image in which a labeled object in a preset image library matches the to-be-recognized object based on the image features. 12.A device for training an object recognition network, comprising: The method comprises: A first obtaining module is configured to obtain a sample image set of attribute information of a labeled sample object; In the to-be-trained object recognition network, the sample image set is divided into at least two image domains based on the attribute information of the sample object and a plurality of preset image domains of a labeled domain label; A first normalization module is configured to normalize a probability that a sample image in the sample image set belongs to each preset image domain to obtain an updated probability set; A second determining module is configured to determine a target updated probability that satisfies a preset condition in the updated probability set; A third determining module is configured to determine a cross-entropy loss between the target updated probability and a probability set as a second loss of dividing the sample image into an image domain; A first determining module is configured to determine a global threshold for evaluating the similarity of the sample object in the at least two image domains and a local threshold for evaluating the similarity of the sample object in each image domain based on a preset misrecognition rate; and a first adjusting module is configured to adjust a classification loss of each image domain in the to-be-trained object recognition network based on the global threshold and the local threshold of each image domain to obtain a first loss. A second adjusting module is configured to adjust the network parameters of the to-be-trained object recognition network based on the first loss and the second loss of the at least two image domains, so that the loss output by the adjusted object recognition network satisfies a preset convergence condition.

13. An object recognition apparatus characterized by comprising: The method comprises: A third obtaining module is configured to obtain a to-be-recognized image including a to-be-recognized object; a first extracting module is configured to perform feature extraction on the to-be-recognized image based on an object recognition network to obtain image features; wherein the object recognition network is trained based on the method of any one of claims 1 to 10; and a first searching module is configured to search for a target preset image in which a labeled object in a preset image library matches the to-be-recognized object based on the image features.

14. A computer storage medium, characterized in that Computer executable instructions are stored in the computer storage medium, and the computer executable instructions can realize the object recognition network training method in any one of claims 1 to 10 after being executed, or the computer executable instructions can realize the object recognition method in claim 11 after being executed.

15. A computer device, comprising: The computer device comprises a memory and a processor, computer executable instructions are stored in the memory, and the processor can realize the object recognition network training method in any one of claims 1 to 10 when running the computer executable instructions in the memory, or the processor can realize the object recognition method in claim 11 when running the computer executable instructions in the memory.

Images