Identity identification method, device and equipment and storage medium

Abstract

The application discloses an identity recognition method and device, equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring a plurality of images containing a to-be-recognized object; performing feature extraction processing on the plurality of images respectively to obtain feature information of each image; determining the confidence degree corresponding to the image based on the feature information of the image; performing fusion processing on the feature information of the images according to the confidence degrees corresponding to the plurality of images respectively to obtain fused feature information; and determining the identity of the to-be-recognized object according to the fused feature information. The application can guarantee the accuracy of identity recognition, avoid inaccurate identity recognition caused by poor quality of a single image, the feature fusion mode is simple, and the identity recognition efficiency is improved.

Description

[0001] This application claims priority from the Chinese Patent Application No. 202110104575.8 filed on January 26, 2021 and entitled "Identity Recognition Method, Device, Equipment and Storage Medium", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of artificial intelligence, in particular to an identity recognition method, device, equipment and storage medium. BACKGROUND

[0003] At present, the identity recognition system is widely used in payment, access control and other practical scenarios.

[0004] In the related art, after the identity recognition system obtains images containing a to-be-recognized object, if the number of images is one image, the identity recognition system directly performs identity recognition on the to-be-recognized object according to the image, and if the number of images is multiple images, the identity recognition system selects one image from the multiple images to perform identity recognition on the to-be-recognized object.

[0005] However, in the above related art, only relying on one image to perform identity recognition on the to-be-recognized object can easily lead to inaccurate recognition results. SUMMARY

[0006] The embodiments of the present application provide an identity recognition method, device, equipment and storage medium, which can ensure the accuracy of identity recognition. The technical solution is as follows:

[0007] According to an aspect of the embodiments of the present application, an identity recognition method is provided, which comprises:

[0008] obtaining multiple images containing a to-be-recognized object;

[0009] respectively performing feature extraction processing on the multiple images to obtain feature information of each image;

[0010] determining a confidence degree corresponding to the image based on the feature information of the image, wherein the confidence degree is used to indicate the reliability of the image in the identity recognition process of the to-be-recognized object;

[0011] performing fusion processing on the feature information of the multiple images according to the confidence degrees respectively corresponding to the multiple images to obtain fusion feature information;

[0012] determining the identity of the to-be-recognized object according to the fusion feature information.

[0013] According to an aspect of some embodiments of the present application, a method for training an identity recognition model is provided, the identity recognition model comprising a feature extraction network, a feature mapping network and a confidence evaluation network, the method comprising:

[0014] obtaining a training sample set, the training sample set comprising sample image sets of at least one identity, each sample image set of an identity comprising a plurality of sample images;

[0015] performing feature extraction processing on the sample images based on the feature extraction network and the feature mapping network after training, to obtain feature information of the sample images;

[0016] obtaining center feature information of the identity based on the feature information of the plurality of sample images of the identity;

[0017] training the confidence evaluation network using the center feature information of the identity and the feature information of the plurality of sample images of the identity, wherein the confidence evaluation network is configured to determine a confidence corresponding to each image of a plurality of images of a to-be-identified object based on feature information of the plurality of images, and the confidence corresponding to each image is used to fuse the feature information of each image to determine an identity of the to-be-identified object;

[0018] generating the identity recognition model comprising the confidence evaluation network.

[0019] According to an aspect of some embodiments of the present application, an identity recognition apparatus is provided, the apparatus comprising:

[0020] an image obtaining module configured to obtain a plurality of images of a to-be-identified object;

[0021] a feature extraction module configured to perform feature extraction processing on the plurality of images respectively to obtain feature information of each image;

[0022] a confidence determination module configured to determine a confidence corresponding to each image based on the feature information of the image, wherein the confidence is used to indicate a reliability of the image in an identity recognition process of the to-be-identified object;

[0023] a feature fusion module configured to perform fusion processing on the feature information of the plurality of images according to the confidences corresponding to the plurality of images respectively, to obtain fusion feature information;

[0024] an identity recognition module configured to determine an identity of the to-be-identified object according to the fusion feature information.

[0025] According to an aspect of some embodiments of the present application, a device for training an identity recognition model is provided, the identity recognition model comprising a feature extraction network, a feature mapping network and a confidence evaluation network, the device comprising:

[0026] a sample obtaining module configured to obtain a training sample set, the training sample set comprising a sample image set of at least one identity, the sample image set of each identity comprising a plurality of sample images;

[0027] a feature obtaining module configured to perform feature extraction processing on the sample images based on the feature extraction network and the feature mapping network after training to obtain feature information of the sample images;

[0028] an information obtaining module configured to obtain center feature information of the identity based on the feature information of the plurality of sample images of the identity;

[0029] a network training module configured to train the confidence evaluation network using the center feature information of the identity and the feature information of the plurality of sample images of the identity, wherein the confidence evaluation network is configured to determine a confidence corresponding to each of a plurality of images of a to-be-identified object based on feature information of the plurality of images, and the confidence corresponding to each of the plurality of images is configured to fuse the feature information of each of the plurality of images to determine an identity of the to-be-identified object;

[0030] a model generating module configured to generate the identity recognition model comprising the confidence evaluation network.

[0031] According to an aspect of some embodiments of the present application, a computer device is provided, the computer device comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the above-mentioned identity recognition method or the above-mentioned training method of the identity recognition model.

[0032] According to an aspect of some embodiments of the present application, a computer readable storage medium is provided, the readable storage medium storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by a processor to implement the above-mentioned identity recognition method or the above-mentioned training method of the identity recognition model.

[0033] According to an aspect of some embodiments of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the identity recognition method or implement the training method of the identity recognition model.

[0034] The technical solutions provided by the embodiments of the present application can bring the following beneficial effects:

[0035] The feature fusion is performed on multiple images containing the to-be-identified object, the identity recognition is performed by using the fused feature information, the accuracy of the identity recognition is ensured, and the inaccurate identity recognition caused by poor quality of a single image is avoided. The feature information of each image is fused according to the confidence degrees corresponding to the multiple images, and the confidence degrees are used to indicate the reliability of the images in the identity recognition process of the to-be-identified object. The reliability of the fused feature information in the identity recognition process is effectively ensured, and the accuracy of the identity recognition is ensured. The confidence degree of each image is determined in units of images for feature fusion. Compared with the related art of obtaining the confidence degrees of features in each dimension for feature fusion, the feature fusion manner is simple, and is beneficial to improving the identity recognition efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0037] Figure 1 is a schematic diagram of a training process and a running process of an identity recognition model provided by an embodiment of the present application;

[0038] Figure 2 is a flowchart of an identity recognition method provided by an embodiment of the present application;

[0039] Figure 3 exemplarily shows a schematic diagram of a flow of identity recognition;

[0040] Figure 4 is a flowchart of a training method of an identity recognition model provided by an embodiment of the present application;

[0041] Figure 5 exemplarily shows a schematic diagram of training of a confidence degree evaluation network;

[0042] Figure 6An exemplary schematic diagram of training of a feature extraction network and a feature mapping network is shown;

[0043] Figure 7 An exemplary schematic diagram of application of the identity recognition manner in the medical field is shown;

[0044] Figure 8 An exemplary schematic diagram of application of the identity recognition manner in the traffic field is shown;

[0045] Figure 9 is a block diagram of an identity recognition device provided by an embodiment of the present application;

[0046] Figure 10 is a block diagram of an identity recognition device provided by another embodiment of the present application;

[0047] Figure 11 is a block diagram of a training device of an identity recognition model provided by an embodiment of the present application;

[0048] Figure 12 is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0050] Artificial intelligence (AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0051] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning and other major directions.

[0052] Computer Vision (CV) is a science that studies how to make machines "see". More specifically, it refers to using cameras and computers to replace human eyes to identify and measure targets, and further process images to make the computer processing more suitable for human observation or transmission to instrument detection. As a scientific discipline, computer vision researches related theories and technologies, and tries to establish artificial intelligence systems that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and other technologies. It also includes common face recognition, fingerprint recognition and other biometric identification technologies.

[0053] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is a subject that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to making computers intelligent. Its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0054] With the research and progress of artificial intelligence technology, artificial intelligence technology has been researched and applied in many fields, such as common smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned vehicles, autonomous vehicles, drones, robots, smart medical care, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0055] The scheme provided by the embodiments of the present application relates to machine learning and other technologies of artificial intelligence, a first training sample set is used to train a feature extraction network and a feature mapping network, and after the training of the feature extraction network and the feature mapping network is completed, the trained feature extraction network and the feature mapping network are used to process a second training sample set, to obtain feature information corresponding to each sample image in the second training sample set, and then the feature information is used to train a confidence evaluation network in combination with central feature information corresponding to each identity. The first training sample set and the second training sample set include at least one sample image set of an identity, and each sample image set of an identity includes a plurality of sample images. Of course, the sample images in the first training sample set and the second training sample set can be the same or different, and the embodiments of the present application do not limit this. After the training of the feature extraction network, the feature mapping network and the confidence evaluation network is completed, the three networks are combined with a feature fusion unit and an identity recognition unit to form an identity recognition model, and the identity recognition model is used to perform identity recognition on an object to be recognized included in a plurality of images. The object to be recognized included in the plurality of images is the same object to be recognized.

[0056] Exemplarily, as shown in the figure, Figure 1 in the training stage of the identity recognition model 10, the first training sample set is used to train the feature extraction network 11 and the feature mapping network 12, and after the training is completed, the second training sample set is used to train the confidence evaluation network 13 on the basis of the trained feature extraction network 11 and the feature mapping network 12; in the use stage of the identity recognition model 10, the feature extraction network 11, the feature mapping network 12, the confidence evaluation network 13 are combined with the feature fusion unit 14 and the identity recognition unit 15 to form the identity recognition model 10, and then after a plurality of images of an object to be recognized are obtained, the identity recognition model 10 is used to perform identity recognition on the object to be recognized in the plurality of images.

[0057] The feature extraction network 11 is used to perform feature extraction processing on an image to obtain a feature map of the image, and the feature map contains spatial structure information of an object to be recognized in the image; the feature mapping network 12 is used to perform dimension reduction processing on the feature to map the feature map into a feature vector; the confidence evaluation network 13 is used to determine a confidence degree corresponding to an image on the basis of a feature map of the image and / or a feature vector of the image, and the confidence degree is directly proportional to the reliability of the image in the process of identity recognition of the object to be recognized; the feature fusion unit 14 is used to fuse feature information of a plurality of images according to confidence degrees corresponding to the plurality of images, to obtain fused feature information, and the feature information includes a feature map and / or a feature vector; and the identity recognition unit 15 is used to determine an identity of an object to be recognized according to the fused feature information.

[0058] It should be noted that the identity recognition model provided in the present application can be widely applied in various fields. Specifically as follows:

[0059] (1) In the field of artificial intelligence, an identity recognition model is set in an artificial intelligence device, so that the artificial intelligence device can process a plurality of images containing a to-be-identified object through the identity recognition model after obtaining the plurality of images, and determine the identity of the to-be-identified object. For example, in the catering field, an identity recognition model can be set in an intelligent service robot, so that the intelligent service robot can identify the identity of a customer, determine whether the customer has made an advance reservation, and guide the customer to a specific location. For another example, in the home field, an identity recognition model can be set in intelligent furniture, so that the intelligent furniture can identify the identity of a furniture user and determine whether the furniture user has the use authority, thereby playing a theft-proof role.

[0060] (2) In the field of transportation, an identity recognition model is set in a face scanning machine at the entrance of various stations, so that the face scanning machine can process a plurality of images containing a to-be-identified object through the identity recognition model after obtaining the plurality of images, and determine the identity of the to-be-identified object.

[0061] (3) In the medical field, an identity recognition model is set in a front desk terminal of a hospital, so that the front desk terminal can determine the identity of a to-be-identified object through the identity recognition model after obtaining a plurality of images containing the to-be-identified object, and then obtain relevant information under the identity of the to-be-identified object, and synchronize the relevant information to a terminal of a medical staff, thereby facilitating the medical staff to provide targeted treatment for the to-be-identified object.

[0062] Of course, the identity recognition model in the present application can also be applied to other various fields, such as person search, conference check-in, warehouse personnel management, etc., which will not be exemplified here.

[0063] For ease of illustration, in the following method embodiments, only the execution subject of each step is taken as a computer device for introduction and illustration. The computer device can be any electronic device with computing and storage capabilities. For example, the computer device can be a server, which can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN (Content Delivery Network), and big data and artificial intelligence platform, etc. For another example, the computer device can also be a terminal, which can be a smartphone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application. It should be noted that in the embodiments of the present application, the execution subject of each step can be the same computer device, or can be executed by multiple different computer devices in cooperation, which is not limited herein. It should also be noted that in the embodiments of the present application, the execution subject of the identity recognition method and the execution subject of the training method of the identity recognition model can be the same computer device, or can be different computer devices, which is not limited in the embodiments of the present application.

[0064] In the following, the technical solutions of the present application will be described in detail in combination with several embodiments.

[0065] Please refer to Figure 2 which shows a flowchart of an identity recognition method provided by an embodiment of the present application. The method can include the following steps (201-205):

[0066] Step 201, obtaining a plurality of images containing a to-be-identified object.

[0067] The to-be-identified object refers to a to-be-detected target with unknown identity. Optionally, the to-be-identified object can be any type of object. For example, the to-be-identified object can be a living object such as a person, a pet, or a wild animal, or a non-living object such as furniture, a machine component, or a utility tool. In the embodiments of the present application, the computer device obtains a plurality of images containing a to-be-identified object when performing identity recognition.

[0068] In one possible implementation, the computer device obtains the plurality of images in real time. Optionally, the computer device obtains the plurality of images for the to-be-identified object from the surrounding environment in real time through a device with a shooting function, and provides the plurality of images to the computer device.

[0069] In another possible implementation, the computer device pre-acquires the to-be-detected images of the to-be-identified object and stores them in the computer device. Alternatively, the computer device acquires the plurality of images by using a device having a photographing function, and stores the plurality of images, and then uniformly performs identity recognition on the plurality of stored images of the to-be-identified object according to a certain time period.

[0070] The device having the photographing function can be directly arranged in the computer device, for example, a camera is arranged in the computer device, or the device having the photographing function can be arranged in an associated device of the computer device, for example, the computer device is a background server, and a camera is arranged in a corresponding front-end terminal.

[0071] Alternatively, in actual application, the computer device can adjust the manner of acquiring the plurality of images containing the to-be-identified object according to actual conditions. For example, in the scenarios of identity verification at a station, identity verification at an examination room, identity verification at an entrance of a public place, and the like, the computer device acquires the plurality of images containing the to-be-identified object in real time, and performs identity recognition on the to-be-identified object contained in the plurality of images, so as to ensure the real-time performance of the identity recognition, and further ensure that the user can timely perform subsequent activities after the identity recognition; in the scenarios of warehouse personnel verification after work starts, identity verification of mobile personnel in a shopping mall, and the like, the real-time performance requirement is not high, and the computer device stores the plurality of acquired images containing the to-be-identified object, and performs identity recognition on the to-be-identified object contained in the plurality of images according to a certain time period.

[0072] Of course, when the computer device acquires the plurality of images, the computer device can directly acquire the plurality of images containing the to-be-identified object, or the computer device can first acquire a to-be-identified video, and then select image frames containing the to-be-identified object from the to-be-identified video, which is not limited in the embodiments of the present application.

[0073] It should be noted that, as for the plurality of images acquired by the computer device, the plurality of images can include the same to-be-identified object, or can include different to-be-identified objects, which is not limited in the embodiments of the present application. If the plurality of images acquired by the computer device contain different to-be-identified objects, the computer device can perform classification processing on the plurality of images, to determine a plurality of image sets, each image set is for a different to-be-identified object, and each image set contains a plurality of images. For example, if the different to-be-identified objects exist in different images, the computer device can directly perform the above classification processing in the unit of image; if the different to-be-identified objects exist in the same image, the computer device can intercept the regions where the different to-be-identified objects are located in the same image, and perform the above classification processing based on the regions.

[0074] In step 202, feature extraction processing is respectively performed on the plurality of images, to obtain feature information of each image.

[0075] In the embodiment of the present application, the computer device performs feature extraction processing on the plurality of images respectively after obtaining the plurality of images, to obtain feature information of each image. The feature information refers to feature information of the to-be-identified object.

[0076] Optionally, the feature information includes a feature map of the image and / or a feature vector of the image. The feature map includes spatial structure information of the to-be-identified object, and the feature vector is obtained through dimension reduction processing on the feature map. In the embodiment of the present application, the computer device obtains the feature information by performing feature extraction processing on the plurality of images respectively, to obtain a plurality of feature maps of the plurality of images, and then performing dimension reduction processing on the feature map of each image, to obtain a plurality of feature vectors of the plurality of images.

[0077] Optionally, the computer device uses a feature extraction network to perform feature extraction processing on the plurality of images respectively, to obtain a feature map of the image, and uses a feature mapping network to map the feature map of the image to a feature vector of the image. The feature extraction network and the feature mapping network are deep neural networks obtained through deep learning training.

[0078] In step 203, the confidence degree corresponding to the image is determined based on the feature information of the image.

[0079] In the embodiment of the present application, the computer device determines the confidence degree corresponding to the image based on the feature information of the image after obtaining the feature information. The confidence degree is used to indicate the reliability degree of the image in the identity recognition process of the to-be-identified object. Optionally, the confidence degree is directly proportional to the reliability degree, that is, the higher the confidence degree corresponding to the image, the higher the reliability degree of the image in the identity recognition process of the to-be-identified object.

[0080] It should be noted that the clarity of the image, the posture of the to-be-identified object in the image, the state of the to-be-identified object in the image, and the like can all affect the confidence degree corresponding to the image. For example, in the case of face-based identity recognition, if the first image is clear, the face is located at the center of the image, and the face is in a front view, and the second image is blurred, the face is located at the corner of the image, and the face is in a side view, then the reliability degree of the first image in the face identity recognition process is higher than that of the second image in the face identity recognition process, that is, the confidence degree of the first image is higher than that of the second image.

[0081] Optionally, the clarity of the image, the posture of the to-be-identified object in the image, the state of the to-be-identified object in the image, and the like are collectively referred to as confidence degree influencing factors. Of course, in actual application, the staff can adjust the confidence degree influencing factors according to the actual situation, and the embodiment of the present application does not limit this.

[0082] In a possible implementation, the feature information includes a feature map of the image and a feature vector of the image, and the computer device determines the confidence corresponding to the image based on the feature map and the feature vector of the image when obtaining the confidence corresponding to the image. Optionally, after obtaining the feature map of the image and the feature vector of the image, the computer device performs confidence evaluation processing in the hyperspherical space based on the feature map and the feature vector of the image by using the confidence evaluation network, to obtain the confidence corresponding to the image; or, after obtaining the feature map of the image and the feature vector of the image, the computer device determines a confidence influence parameter corresponding to the image based on the feature map and the feature vector of the image, and then determines the confidence corresponding to the image according to the confidence influence parameter.

[0083] In another possible implementation, the feature information includes a feature map of the image, and the computer device determines the confidence corresponding to the image by using the feature map of the image when obtaining the confidence of the image. Optionally, after obtaining the feature map of the image, the computer device performs confidence evaluation processing in the hyperspherical space based on the feature map of the image by using the confidence evaluation network, to obtain the confidence corresponding to the image; or, after obtaining the feature map of the image, the computer device determines a confidence influence parameter corresponding to the image based on the feature map of the image, and then determines the confidence corresponding to the image according to the confidence influence parameter.

[0084] In another possible implementation, the feature information includes a feature map of the image, and the computer device determines the confidence corresponding to the image by using the feature map of the image when obtaining the confidence of the image. Optionally, after obtaining the feature map of the image, the computer device performs confidence evaluation processing in the hyperspherical space based on the feature map of the image by using the confidence evaluation network, to obtain the confidence corresponding to the image; or, after obtaining the feature map of the image, the computer device determines a confidence influence parameter corresponding to the image based on the feature map of the image, and then determines the confidence corresponding to the image according to the confidence influence parameter.

[0085] In a possible implementation, the feature information includes a feature map of the image and a feature vector of the image, and the computer device determines the confidence corresponding to the image based on the feature map and the feature vector of the image when obtaining the confidence corresponding to the image. Optionally, after obtaining the feature map of the image and the feature vector of the image, the computer device performs confidence evaluation processing in the hyperspherical space based on the feature map and the feature vector of the image by using the confidence evaluation network, to obtain the confidence corresponding to the image; or, after obtaining the feature map of the image and the feature vector of the image, the computer device determines a confidence influence parameter corresponding to the image based on the feature map and the feature vector of the image, and then determines the confidence corresponding to the image according to the confidence influence parameter.

[0086] It should be noted that, since the feature vector is obtained from the feature map, in the case where the feature information only includes the feature map, the computer device can also determine the confidence corresponding to the image by using the feature vector of the image.

[0087] It should be noted that, in the embodiments of the present application, the feature information is obtained by the feature extraction network and the feature mapping network, and is processed in the hyperspherical space, so that the matching degree between the processing results of the three networks can be ensured, and the error caused by different processing spaces can be avoided.

[0088] At step 204, the feature information of the multiple images is fused according to the confidence degrees respectively corresponding to the multiple images, to obtain fused feature information.

[0089] In the embodiments of the present application, after obtaining the confidence degrees respectively corresponding to the multiple images, the computer device fuses the feature information of the multiple images according to the confidence degrees respectively corresponding to the multiple images, to obtain fused feature information. Optionally, the computer device can determine the fusion weights respectively corresponding to the multiple images based on the confidence degrees respectively corresponding to the multiple images, and then perform weighted summation processing on the feature information of the multiple images based on the fusion weights respectively corresponding to the multiple images, to obtain the fused feature information. Optionally, the fused feature information includes a fused feature vector.

[0090] In a possible implementation, the feature information includes a feature map. After obtaining the confidence degrees respectively corresponding to the multiple images, the computer device fuses the feature maps of the multiple images according to the confidence degrees respectively corresponding to the multiple images, to obtain a fused feature map, and then maps the fused feature map to a fused feature vector.

[0091] In another possible implementation, the feature information includes a feature vector. After obtaining the confidence degrees respectively corresponding to the multiple images, the computer device fuses the feature vectors of the multiple images according to the confidence degrees respectively corresponding to the multiple images, to obtain a fused feature vector. The feature vector is a vector representation obtained by performing dimension reduction processing on a feature map of an image by using a feature mapping network, and the feature map is a feature image containing spatial structure information of the object to be recognized, which is obtained by using a feature extraction network.

[0092] For example, assuming that the uncertainty of an image is k (the confidence degree is 1-k), the feature vector of the image is μ, and the number of images containing the object to be recognized is m, the fused feature vector z is:

[0093]

[0094] It should be noted that, since the feature vector is obtained from the feature map, in the case where the feature information only includes the feature map, the computer device can also obtain the fused feature vector from the feature vector of the image.

[0095] In step 205, the identity of the object to be identified is determined according to the fused feature information.

[0096] In the embodiments of the present application, after the computer device acquires the fused feature information, the identity of the object to be identified is determined according to the fused feature information. Optionally, the computer device determines the similarity between the object to be identified and the target object according to the fused feature information, and if the similarity is less than a threshold, it is determined that the identity of the object to be identified is the same as the identity of the target object.

[0097] Optionally, if the fused feature information is a fused feature vector, after the computer device acquires the fused feature vector, it can determine the distance between the fused feature vector and the feature vector of the target object according to the fused feature vector, and if the distance is less than a threshold, it is determined that the identity of the object to be identified is the same as the identity of the target object. The feature vector of the target object refers to a feature vector that can represent the identity of the target object.

[0098] The target object refers to an object with known identity, and optionally, the target object can be pre-stored in the computer device. Of course, in actual application, the staff can adjust the storage of the target object according to the actual situation, for example, if the identity recognition method of the present application is applied to the search of a single object, the computer device can only store the object to be searched as the target object.

[0099] In summary, in the technical scheme provided by the embodiments of the present application, the features of multiple images containing the object to be identified are fused, and the fused feature information is used for identity recognition, which ensures the accuracy of identity recognition and avoids inaccurate identity recognition caused by poor quality of a single image. The feature information of each image is fused according to the confidence degrees corresponding to the multiple images, and the confidence degree is used to indicate the reliability of the image in the identity recognition process of the object to be identified, which can effectively ensure the reliability of the fused feature information in the identity recognition process and ensure the accuracy of identity recognition. The confidence degree of each image is determined in units of images for feature fusion, which is simpler than the feature fusion of the confidence degrees of features in each dimension in related technologies, and is conducive to improving the efficiency of identity recognition.

[0100] Optionally, in the embodiment of the present application, the identity recognition method is implemented by an identity recognition model, which comprises a feature extraction network, a feature mapping network, a confidence evaluation network, a feature fusion unit and an identity recognition unit. The feature extraction network is configured to perform feature extraction processing on the plurality of images respectively to obtain feature maps of the images; the feature mapping network is configured to map the feature maps of the images into feature vectors of the images; the confidence evaluation network is configured to determine the confidence degrees corresponding to the images based on the feature maps and the feature vectors of the images; the feature fusion unit is configured to perform fusion processing on the feature information of the plurality of images according to the confidence degrees corresponding to the plurality of images respectively to obtain fused feature information, which comprises the feature maps and / or the feature vectors; and the identity recognition unit is configured to determine the identity of the to-be-recognized object according to the fused feature information.

[0101] Exemplarily, in combination with reference to Figure 3 The way of identity recognition by using the identity recognition model is introduced. After the computer device obtains the plurality of images containing the to-be-recognized object, the feature extraction network 31 is used to perform feature extraction processing on the plurality of images respectively to obtain the feature maps of the plurality of images, and then the feature mapping network 32 is used to perform dimension reduction processing on the feature maps of the plurality of images respectively to map the feature maps into feature vectors, thereby obtaining the feature vectors of the plurality of images. Then, the confidence evaluation network 33 is used to determine the confidence degrees corresponding to the plurality of images based on the feature maps and the feature vectors of the images respectively, and then the feature fusion unit 34 is used to perform feature fusion on the feature vectors of the images according to the confidence degrees corresponding to the plurality of images respectively to obtain fused feature vectors, and the identity recognition unit 35 is used to determine the identity of the to-be-recognized object according to the distance between the fused feature vectors and the feature vector of the target object. The feature vector of the target object refers to a feature vector that can indicate the identity of the target object, and the target object can include one or more objects whose identities are known.

[0102] Next, the training method of the identity recognition model in the present application is introduced.

[0103] Please refer to Figure 4 which shows a flowchart of the training method of the identity recognition model according to an embodiment of the present application. The identity recognition model comprises a feature extraction network, a feature mapping network and a confidence evaluation network. The method can comprise the following steps (401-405):

[0104] Step 401, obtaining a training sample set.

[0105] The training sample set includes sample image sets for at least one identity, and each identity's sample image set includes multiple sample images. Each sample image for each identity includes a sample object corresponding to that identity. Exemplarily, the sample object can be any type of object. Exemplarily, the sample object can be a living object such as a person, pet, or wild animal, or a non-living object such as furniture, machine components, or utility tools; this application embodiment does not limit this. In this application embodiment, when a computer device determines to train an identity recognition model, it acquires a training sample set for that identity recognition model.

[0106] In one possible implementation, the computer device collects the sample images by taking physical photographs, thereby generating a training sample set. In another possible implementation, the computer device collects the sample images from a network environment, thereby generating a training sample set. Of course, in practical applications, the method of collecting the sample images can be determined according to the actual situation. For example, in the medical field, the computer device can acquire multiple images of various users who have visited the hospital as the sample images, and subsequently acquire images of users who have visited the hospital for the first time as sample images, thereby updating and training the identity recognition model so that the identity recognition model can recognize more user identities.

[0107] Step 402: Based on the trained feature extraction network and feature mapping network, perform feature extraction processing on the sample image to obtain the feature information of the sample image.

[0108] In this embodiment, after acquiring the training sample set, the computer device performs feature extraction processing on the sample images based on the trained feature extraction network and feature mapping network to obtain the feature information of the sample images. The feature extraction network and feature mapping network are deep neural networks obtained through deep learning, and the feature information refers to the feature information of the sample objects contained in the sample images.

[0109] Optionally, the aforementioned feature information includes the feature map and feature vector of the sample image. After acquiring the training sample set, the computer device uses a trained feature extraction network to perform feature extraction processing on the sample images, obtaining the feature map of the sample images. Then, it uses a trained feature mapping network to perform dimensionality reduction processing on the feature map of the sample images, mapping the feature map of the sample images into feature vectors of the sample images. The aforementioned feature map includes the spatial structure information of the sample objects contained in the sample image.

[0110] Step 403: Based on the feature information of multiple sample images of the identity, obtain the central feature information of the identity.

[0111] In the embodiments of the present application, the computer device obtains the central feature information of the identity based on the feature information of the plurality of sample images of the identity before training the confidence evaluation network, wherein the central feature information refers to the feature information capable of representing the identity.

[0112] In a possible implementation, after obtaining the training sample set, the computer device selects a plurality of standard images from the plurality of sample images of the identity, and performs feature extraction processing on the plurality of standard images of the identity based on the feature extraction network and the feature mapping network after training, to obtain the feature information of the plurality of standard images, and then performs average processing on the feature information of the plurality of standard images to obtain the central feature information of the identity. The standard image refers to an image capable of clearly representing the identity of the sample object. Alternatively, the computer device can select the standard image from the sample image according to a preset rule. For example, the computer device takes the resolution of the sample image as the selection criterion for the standard image, such as determining the sample image with a resolution greater than a target value as the standard image; or the computer device takes the posture of the sample object in the sample image as the selection criterion for the standard image, such as determining the front view sample image of the sample object as the standard image; and the like.

[0113] In another possible implementation, after obtaining the training sample set, the computer device performs feature extraction processing on the plurality of sample images of the identity based on the feature extraction network and the feature mapping network after training, to obtain the feature information of the plurality of sample images, and determines the classification weight of the feature information of the plurality of sample images, and then selects the feature information with the largest classification weight as the central feature information of the identity. The classification weight is used to indicate the closeness between the feature information of the sample image and the identity corresponding to the sample image, and the classification weight is directly proportional to the closeness.

[0114] In step 404, the confidence evaluation network is trained using the central feature information of the identity and the feature information of the plurality of sample images of the identity.

[0115] In the embodiments of the present application, the computer device trains the confidence evaluation network using the central feature information of the identity and the feature information of the plurality of sample images of the identity after obtaining the central feature information and the feature information. The confidence evaluation network is used to determine the confidence corresponding to each image based on the feature information of the plurality of images of the to-be-recognized object, and the confidence corresponding to each image is used to fuse the feature information of each image to determine the identity of the to-be-recognized object.

[0116] Alternatively, the step 404 includes the following steps:

[0117] 1. The confidence evaluation network is used to evaluate the feature information of the identity-based sample image to obtain the confidence of the identity-based sample image.

[0118] 2. The loss function of the confidence evaluation network is determined based on the central feature information of the identity, the feature information of the multiple sample images of the identity, and the confidence of the multiple sample images of the identity.

[0119] 3. The parameters of the confidence evaluation network are adjusted based on the loss function.

[0120] In the embodiments of the present application, when the computer device trains the confidence evaluation network, the confidence evaluation network is used to evaluate the feature information of the identity-based sample image to obtain the confidence of the identity-based sample image, and then the loss function of the confidence evaluation network is determined based on the central feature information of the identity, the feature information of the multiple sample images of the identity, and the confidence of the multiple sample images of the identity. The gradient function of the confidence evaluation network is determined based on the loss function, and the parameters of the confidence evaluation network are adjusted in the gradient descent manner based on the gradient function until the training iteration number of the confidence evaluation network exceeds the target value or the loss function converges, and the training of the confidence evaluation network is completed.

[0121] For example, assuming that the uncertainty of the sample image is k (the confidence is 1-k), the feature vector of the sample image is μ, the dimension of the feature vector μ is d, the radius of the feature vector μ in the hypersphere space is r, and the central feature vector of the identity is w, then the loss function L of the confidence evaluation network is: s

[0122]

[0123]

[0124] wherein x refers to the sample image input to the confidence evaluation network, c refers to the identity corresponding to the sample image, I refers to the Bessel function, α refers to the order of the Bessel function, and the hypersphere space refers to the training space of the feature extraction network, the feature mapping network, and the confidence evaluation network.

[0125] Further, the computer device determines the gradient function s of the confidence evaluation network based on the loss function L and are respectively:

[0126]

[0127]

[0128] ​Afterwards, the computer device adjusts the parameters of the confidence evaluation network in a gradient descent manner until the confidence evaluation network is trained.

[0129] It should be noted that the loss function L s The above is described from the perspective of a single sample image. In actual use, the computer device can obtain the loss function for each sample image when training the confidence evaluation network, and obtain the final loss function by summing and averaging each loss function.

[0130] For example, with reference to Figure 5 The training method of the confidence evaluation network is described. In the preparation stage of the training data, the feature extraction network trained and the feature mapping network trained are used to perform feature extraction processing on the sample images in the training sample set to obtain the feature information of the sample images and obtain the center feature information of the identity. The feature information includes feature maps and feature vectors. Then, the confidence evaluation network is used to determine the confidence corresponding to the sample image based on the feature information, and to determine the loss function of the confidence evaluation network according to the confidence corresponding to the sample image, the feature vector of the sample image and the center feature information of the identity, and to determine whether the confidence evaluation network meets the training completion condition based on the loss function. If the confidence evaluation network meets the training completion condition, it is determined that the confidence evaluation network is trained; if the confidence evaluation network does not meet the training completion condition, a new training sample set is obtained, and the above steps are repeated to continue training the confidence evaluation network.

[0131] Step 405, generating an identity recognition model including the confidence evaluation network.

[0132] In the embodiments of the present application, the computer device generates an identity recognition model including the confidence evaluation network after determining that the confidence evaluation network is trained. Optionally, the identity recognition model further includes a feature fusion unit and an identity recognition unit. The feature fusion unit is configured to perform fusion processing on the feature information of multiple images according to the confidence corresponding to each of the multiple images to obtain fusion feature information, and the feature information includes feature maps and / or feature vectors. The identity recognition unit is configured to determine the identity of the to-be-identified object contained in the multiple images according to the fusion feature information.

[0133] In summary, in the technical scheme provided by the embodiments of the present application, the identity recognition model comprises a confidence evaluation network, and the confidence evaluation network is configured to determine the confidence of each image according to the feature information of the plurality of images of the to-be-identified object, wherein the confidence is used to fuse the feature information of each image to determine the identity of the to-be-identified object. Thus, in the identity recognition process, the identity recognition model can fuse the features of the plurality of images according to the confidence of each image, and perform identity recognition by using multiple images, thereby ensuring the accuracy of identity recognition and avoiding inaccurate identity recognition caused by poor quality of a single image.

[0134] In addition, in combination with reference Figure 6 The training process of the feature extraction network and the feature mapping network is introduced. The specific steps are as follows:

[0135] In step 601, a training sample set for the feature extraction network and the feature mapping network is obtained. The training sample set comprises at least one sample image with an annotated identity. Moreover, the training sample set for the feature extraction network and the feature mapping network can be the same training sample set as the training sample set for the confidence evaluation network, or can be a different training sample set, which is not limited in the embodiments of the present application.

[0136] In step 602, the sample images are input into the feature extraction network respectively, and the feature maps of the plurality of sample images are obtained.

[0137] In step 603, the feature maps of the plurality of sample images are input into the feature mapping network respectively, and the feature vectors of the plurality of sample images are obtained.

[0138] In step 604, the loss function of the feature extraction network and the feature mapping network is determined according to the feature vectors of the plurality of sample images and the annotated identities of the plurality of sample images. The feature function is used to represent the difference between the feature vectors output by the feature mapping network and the actual annotated identities.

[0139] In step 605, whether the feature extraction network and the feature mapping network satisfy the training completion condition is determined according to the loss function of the feature extraction network and the feature mapping network. If the feature extraction network and the feature mapping network satisfy the training completion condition, the process is ended; if the feature extraction network and the feature mapping network do not satisfy the training completion condition, step 606 is performed. The training completion condition can be that the number of iterations of the feature extraction network and the feature mapping network exceeds a required value, or the loss function of the feature extraction network and the feature mapping network converges.

[0140] In step 606, a new training sample set is obtained, and steps 601-605 are repeated.

[0141] Of course, in the embodiment of the present application, the identity recognition model further includes: a feature fusion unit and an identity recognition unit. The feature fusion unit is configured to perform fusion processing on the feature information of the plurality of images according to the confidence degrees respectively corresponding to the plurality of images, to obtain fused feature information, the feature information including feature maps and / or feature vectors; and the identity recognition unit is configured to determine the identity of the to-be-recognized object contained in the plurality of images according to the fused feature information.

[0142] In the following, Figure 7 The identity recognition method in the present application can be applied in the medical field. Alternatively, the execution subject of the identity recognition method is a front desk terminal of a hospital, and the specific steps are as follows:

[0143] In step 701, n face images of a to-be-identified patient are obtained.

[0144] In the embodiment of the present application, when determining the identity of the to-be-identified patient, the front desk terminal can first obtain n face images of the to-be-identified patient. n is a positive integer greater than one, of course, the staff can also flexibly adjust the value of n according to the actual situation, and the embodiment of the present application does not limit this.

[0145] Alternatively, the front desk terminal can obtain n face images of the to-be-identified patient according to an electronic device with a shooting function. The electronic device with a shooting function can be directly arranged on the front desk terminal, or can be connected to the front desk terminal through a network, and the embodiment of the present application does not limit this.

[0146] In step 702, a feature extraction network is used to extract features of the n face images respectively, to obtain feature maps of the n face images.

[0147] In the embodiment of the present application, after obtaining the n face images, the front desk terminal uses a feature extraction network to extract features of the n face images respectively, to obtain feature maps of the n face images. The feature maps include spatial structure information of the face.

[0148] In step 703, a feature mapping network is used to perform dimension reduction processing on the feature maps of the n face images respectively, to obtain n feature vectors.

[0149] In the embodiment of the present application, after obtaining the n feature maps, the front desk terminal uses a feature mapping network to perform dimension reduction processing on the feature maps of the n face images respectively, to obtain n feature vectors.

[0150] In step 704, a confidence degree evaluation network is used to obtain confidence degrees corresponding to the n face images based on the n feature maps and the n feature vectors.

[0151] In the embodiment of the present application, after the n feature maps and the n feature vectors are obtained, the foreground terminal obtains the confidence corresponding to the n face images based on the n feature maps and the n feature vectors by using the confidence evaluation network. The confidence is directly proportional to the reliability, that is, the higher the confidence corresponding to the face image, the higher the reliability of the face image in the process of identifying the identity of the patient to be identified.

[0152] Optionally, in the embodiment of the present application, the shooting angle, the shooting light, and the like of the patient to be identified can affect the confidence of the face image. Therefore, in order to ensure the accuracy of the collection of the face image, the staff can adjust the placement position of the foreground terminal according to the actual situation, and set voice prompt information to remind the patient to be identified to take a photo at a suitable position or at a suitable angle.

[0153] In step 705, the feature fusion unit fuses the feature vectors of the n face images according to the confidence corresponding to each of the n face images, to obtain fusion feature information.

[0154] In the embodiment of the present application, after the confidence is obtained, the feature fusion unit fuses the feature vectors of the n face images according to the confidence corresponding to each of the n face images, to obtain fusion feature information. Optionally, the fusion feature information includes a fusion feature vector.

[0155] In step 706, the identity recognition unit determines the identity of the patient to be identified according to the fusion feature information.

[0156] In the embodiment of the present application, after the fusion feature information is obtained, the identity recognition unit determines the identity of the patient to be identified according to the fusion feature information.

[0157] Optionally, the fusion feature information includes a fusion feature vector. When the identity of the patient to be identified is determined, the feature vector of each patient is obtained from the patient information database, and the distance between the feature vector of each patient and the fusion feature vector is determined, so that in the case where the distance is less than a threshold value, the patient corresponding to the threshold value is determined to be the patient to be identified, and the identity recognition of the patient to be identified is realized. The patient information database can be independently stored in a hospital or jointly stored in hospitals, which is not limited in the embodiment of the present application.

[0158] In step 707, related information is obtained based on the identity of the patient to be identified.

[0159] Optionally, in the embodiment of the present application, after the identity of the patient to be identified is determined, the related information is obtained based on the identity of the patient to be identified.

[0160] In a possible implementation, information of different hospitals is independently stored and not communicated with each other, and the related information can be information stored in the hospital.

[0161] In another possible implementation, information of different hospitals is stored together or communicated with each other, and the related information can be related information stored in other hospitals.

[0162] Optionally, after obtaining the related information, the front desk terminal sends the related information to the user terminal of the doctor, so that the doctor can learn about the to-be-identified patient.

[0163] In summary, by performing feature fusion on multiple images containing a to-be-identified object, and using fused feature information to perform identity recognition, the accuracy of identity recognition is ensured, and inaccurate identity recognition caused by poor quality of a single image is avoided. According to the confidence degrees of the multiple images respectively corresponding to the feature information of each image, and the confidence degrees being used to indicate the reliability of the images in the identity recognition process of the to-be-identified object, the reliability of the fused feature information in the identity recognition process is effectively ensured, and the accuracy of identity recognition is ensured. In the medical field, the identity recognition scheme is used, and the related information of the user is determined according to the identity of the to-be-identified patient, so that the doctor can sufficiently learn about the related information of the patient, and the effectiveness and pertinence of treatment are ensured.

[0164] In the following, reference is made to Figure 8 The identity recognition method in this application can be applied in the transportation field. Optionally, the execution subject of the identity recognition method is a face scanning terminal at a station entrance, and the specific steps are as follows:

[0165] Step 801: Obtain p face images containing a to-be-identified object.

[0166] In the embodiment of the application, when determining the identity of the to-be-identified object, the face scanning terminal can first obtain p face images containing the to-be-identified object. Wherein, p is a positive integer greater than one, of course, the staff can also flexibly adjust the value of p according to the actual situation, and the embodiment of the application does not limit this.

[0167] Optionally, the face scanning terminal can obtain p face images containing the to-be-identified object according to an electronic device with a shooting function. Wherein, the electronic device with a shooting function can be directly arranged on the face scanning terminal, or can be connected to the face scanning terminal through a network, and the embodiment of the application does not limit this.

[0168] Step 802: Use a feature extraction network to perform feature extraction on the p face images respectively, to obtain feature maps of the p face images.

[0169] In this embodiment, after acquiring the aforementioned p face images, the face scanning terminal uses a feature extraction network to extract features from each of the p face images, obtaining feature maps of the p face images. These feature maps include spatial structural information of the faces.

[0170] Step 803: Use a feature mapping network to reduce the dimensionality of the feature maps of the p face images to obtain p feature vectors.

[0171] In this embodiment of the application, after acquiring the above p feature maps, the face scanning terminal uses a feature mapping network to perform dimensionality reduction processing on the feature maps of the p face images respectively, to obtain p feature vectors.

[0172] Step 804: Use a confidence evaluation network based on p feature maps and p feature vectors to obtain the confidence scores of p face images.

[0173] In this embodiment, after acquiring the p feature maps and p feature vectors, the face scanning terminal uses a confidence evaluation network to obtain the confidence scores of p face images based on the p feature maps and p feature vectors. The confidence score is directly proportional to the reliability; that is, the higher the confidence score of a face image, the higher the reliability of the face image in the process of identifying the identity of the object to be identified.

[0174] Optionally, in this embodiment, the shooting angle and lighting of the object to be identified can affect the confidence level of the face image. Therefore, in order to ensure the accuracy of face image acquisition, staff can adjust the placement of the face scanning terminal according to the actual situation and set voice prompts to remind the object to be identified to take the photo from a suitable position or angle.

[0175] Step 805: The feature fusion unit performs feature fusion on the feature vectors of the p face images based on the confidence levels corresponding to the p face images respectively, and obtains fused feature information.

[0176] In this embodiment, after obtaining the aforementioned confidence level, the face scanning terminal uses a feature fusion unit to perform feature fusion on the feature vectors of the p face images based on the confidence levels corresponding to each of the p face images, thereby obtaining fused feature information. Optionally, the fused feature information includes fused feature vectors.

[0177] Step 806: The identity recognition unit determines the identity of the object to be identified based on the fused feature information.

[0178] In this embodiment of the application, after obtaining the above-mentioned fused feature information, the face scanning terminal uses an identity recognition unit to determine the identity of the object to be identified based on the fused feature information.

[0179] Optionally, the fusion feature information comprises a fusion feature vector. The face scanning terminal determines the feature vectors of the users who purchased the tickets from the ticket information library, and determines the distance between the feature vectors of the users and the fusion feature vector, and then determines the user corresponding to the threshold as the to-be-identified object when the distance is less than the threshold, so as to realize the identity recognition of the to-be-identified object.

[0180] In summary, the features of multiple images containing the to-be-identified object are fused, the fusion feature information is used for identity recognition, the accuracy of identity recognition is ensured, and the inaccurate identity recognition caused by poor quality of a single image is avoided; the feature information of each image is fused according to the confidence degrees corresponding to the multiple images, and the confidence degrees are used to indicate the reliability of the images in the identity recognition process of the to-be-identified object, so that the reliability of the fusion feature information in the identity recognition process is effectively ensured, and the accuracy of identity recognition is ensured; the identity recognition scheme is applied in the transportation field, and the station management is better ensured under the condition of ensuring the accuracy of identity recognition.

[0181] It should be noted that in the specific embodiments of the present application, the user information related data such as images, face images, related information and the like involved in the present application need to obtain the permission or consent of the user when the above embodiments of the present application are applied to specific products or technologies, and the collection, use and processing of the related data need to comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0182] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the method embodiments of the present application.

[0183] Please refer to Figure 9 which shows a block diagram of an identity recognition apparatus provided by an embodiment of the present application. The apparatus has the functions of implementing the above identity recognition method, which can be implemented by hardware or corresponding software executed by hardware. The apparatus can be a computer device or can be arranged in a computer device. The apparatus 900 can include an image acquisition module 910, a feature extraction module 920, a confidence degree determination module 930, a feature fusion module 940 and an identity recognition module 950.

[0184] The image acquisition module 910 is configured to acquire multiple images containing a to-be-identified object.

[0185] The feature extraction module 920 is configured to perform feature extraction processing on the multiple images respectively, to obtain feature information of each of the images.

[0186] The confidence determination module 930 is configured to determine a confidence corresponding to the image based on the feature information of the image, where the confidence is used to indicate a reliability of the image in an identity recognition process of the to-be-identified object.

[0187] The feature fusion module 940 is configured to perform fusion processing on the feature information of the plurality of images according to the confidence corresponding to each of the plurality of images, to obtain fusion feature information.

[0188] The identity recognition module 950 is configured to determine the identity of the to-be-identified object according to the fusion feature information.

[0189] In an example embodiment, the confidence determination module 930 is configured to determine the confidence corresponding to the image based on a feature map and a feature vector of the image, where the feature map is a feature image containing spatial structure information of the to-be-identified object obtained by using a feature extraction network, and the feature vector is a vector representation obtained by performing dimension reduction processing on the feature map of the image by using a feature mapping network.

[0190] In an example embodiment, the confidence determination module 930 is further configured to perform confidence evaluation processing in a hyperspherical space based on the feature map and the feature vector of the image by using a confidence evaluation network, to obtain the confidence corresponding to the image.

[0191] In an example embodiment, the fusion feature information includes a fusion feature vector; the feature fusion module 940 is configured to perform fusion processing on feature maps of the plurality of images according to the confidence corresponding to each of the plurality of images, to obtain a fusion feature map; and map the fusion feature map into a fusion feature vector; where the feature map is a feature image containing spatial structure information of the to-be-identified object obtained by using a feature extraction network; or perform fusion processing on feature vectors of the plurality of images according to the confidence corresponding to each of the plurality of images, to obtain a fusion feature vector; where the feature vector is a vector representation obtained by performing dimension reduction processing on the feature map of the image by using a feature mapping network, and the feature map is a feature image containing spatial structure information of the to-be-identified object obtained by using a feature extraction network.

[0192] In an example embodiment, as shown in Figure 10 The feature fusion module 940 includes a weight determination unit 941 and a feature acquisition unit 942.

[0193] The weight determination unit 941 is configured to determine fusion weights corresponding to the plurality of images based on the confidence corresponding to each of the plurality of images.

[0194] The feature acquisition unit 942 is configured to perform weighted summation processing on feature information of the plurality of images based on the fusion weights corresponding to the plurality of images respectively, to obtain the fusion feature information.

[0195] In an example embodiment, the method is implemented by an identity recognition model, the method is implemented by the identity recognition model, and the identity recognition model includes a feature extraction network, a feature mapping network, a confidence evaluation network, a feature fusion unit, and an identity recognition unit. The feature extraction network is configured to perform feature extraction processing on the plurality of images respectively to obtain feature maps of the images. The feature mapping network is configured to map the feature maps of the images into feature vectors of the images. The confidence evaluation network is configured to determine confidences corresponding to the images based on the feature maps and the feature vectors of the images. The feature fusion unit is configured to perform fusion processing on feature information of the plurality of images according to the confidences corresponding to the images respectively to obtain the fusion feature information. The feature information includes the feature maps and / or the feature vectors. The identity recognition unit is configured to determine the identity of the to-be-recognized object according to the fusion feature information.

[0196] In summary, in the technical scheme provided by the embodiments of the present application, feature fusion is performed on a plurality of images containing a to-be-recognized object, and identity recognition is performed using fusion feature information, thereby ensuring the accuracy of identity recognition and avoiding inaccurate identity recognition caused by poor quality of a single image. The feature information of each image is fused according to the confidences corresponding to the images respectively, and the confidences are used to indicate the reliability of the images in the process of identity recognition of the to-be-recognized object, thereby effectively ensuring the reliability of the fusion feature information in the process of identity recognition and ensuring the accuracy of identity recognition. The confidences of each image are determined in units of images for feature fusion, which is simpler than the related art of obtaining confidences of features in each dimension for feature fusion, and is conducive to improving the efficiency of identity recognition.

[0197] For reference Figure 11 which shows a block diagram of a training device of an identity recognition model provided by an embodiment of the present application. The identity recognition model includes a feature extraction network, a feature mapping network, and a confidence evaluation network. The device has the function of implementing the training method of the above-mentioned identity recognition model, which can be implemented by hardware, or the corresponding software can be executed by hardware. The device can be a computer device, or can be arranged in a computer device. The device 1100 can include a sample acquisition module 1110, a feature acquisition module 1120, an information acquisition module 1130, and a network training module 1140.

[0198] The sample obtaining module 1110 is configured to obtain a training sample set, wherein the training sample set comprises a sample image set of at least one identity, and each sample image set of an identity comprises a plurality of sample images.

[0199] The feature obtaining module 1120 is configured to perform feature extraction processing on the sample images based on the trained feature extraction network and the feature mapping network, to obtain feature information of the sample images.

[0200] The information obtaining module 1130 is configured to obtain the center feature information of the identity based on the feature information of the plurality of sample images of the identity.

[0201] The network training module 1140 is configured to train the confidence evaluation network by using the center feature information of the identity and the feature information of the plurality of sample images of the identity, wherein the confidence evaluation network is configured to determine a confidence degree corresponding to each image of a to-be-identified object based on feature information of the plurality of images, and the confidence degree corresponding to each image is configured to fuse the feature information of each image to determine the identity of the to-be-identified object.

[0202] The model generating module 1150 is configured to generate the identity recognition model comprising the confidence evaluation network.

[0203] In exemplary embodiments, the network training module 1140 is configured to perform confidence evaluation processing on the sample images of the identity based on the feature information of the sample images by using the confidence evaluation network, to obtain a confidence degree of the sample images of the identity; determine a loss function of the confidence evaluation network based on the center feature information of the identity, the feature information of the plurality of sample images of the identity, and the confidence degrees of the plurality of sample images of the identity; and adjust parameters of the confidence evaluation network based on the loss function.

[0204] In exemplary embodiments, the feature obtaining module 1120 is configured to select a plurality of standard images from the plurality of sample images of the identity; perform feature extraction processing on the plurality of standard images of the identity based on the trained feature extraction network and the feature mapping network, to obtain feature information of the plurality of standard images; perform average processing on the feature information of the plurality of standard images, to obtain the center feature information of the identity; or perform feature extraction processing on the plurality of sample images of the identity based on the trained feature extraction network and the feature mapping network, to obtain feature information of the plurality of sample images; determine classification weights of the feature information of the plurality of sample images; select feature information with the largest classification weight as the center feature information of the identity; wherein the classification weight is used to indicate a closeness between the feature information of the sample images and the identity, and the classification weight is directly proportional to the closeness.

[0205] In an example embodiment, the information obtaining module 1130 is configured to perform feature extraction processing on the sample image by using the trained feature extraction network to obtain a feature map of the sample image, the feature map comprising spatial structure information of a sample object contained in the sample image; and map the feature map of the sample image to a feature vector of the sample image by using the trained feature mapping network; wherein the feature information comprises the feature map of the sample image and the feature vector of the sample image.

[0206] In an example embodiment, the confidence evaluation network is applied to an identity recognition model, and the identity recognition model further comprises a feature extraction network, a feature mapping network, a feature fusion unit and an identity recognition unit; wherein the feature extraction network is configured to perform feature extraction processing on a plurality of images respectively to obtain a feature map of each of the images; the feature mapping network is configured to map the feature map of the image to a feature vector of the image; the feature fusion unit is configured to perform fusion processing on feature information of the plurality of images according to the confidence degrees respectively corresponding to the plurality of images to obtain fused feature information; wherein the feature information comprises a feature map and / or a feature vector; and the identity recognition unit is configured to determine the identity of a to-be-recognized object contained in the plurality of images according to the fused feature information.

[0207] In summary, in the technical scheme provided by the embodiments of the present application, the identity recognition model comprises a confidence evaluation network, and the confidence evaluation network is configured to determine the confidence degree corresponding to each image according to feature information of a plurality of images of a to-be-recognized object, wherein the confidence degree is used to fuse the feature information of each image to determine the identity of the to-be-recognized object, so that the identity recognition model can fuse the feature information of the plurality of images according to the confidence degrees of the images in the identity recognition process, on the one hand, the identity is recognized by using multiple images to ensure the accuracy of the identity recognition and avoid inaccurate identity recognition caused by poor quality of a single image.

[0208] It should be noted that the apparatus provided in the above embodiments is only used as an example to illustrate the division of the above functional modules in realizing the functions thereof, and in actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0209] Please refer to Figure 12FIG. 12 shows a structural block diagram of a computer device according to an embodiment of the present application. The computer device can be used to implement the functions of the above-mentioned identity recognition method or the training method of the identity recognition model. Specifically,

[0210] The computer device 1200 includes a central processing unit (CPU) 1201, a system memory 1204 including a random access memory (RAM) 1202 and a read-only memory (ROM) 1203, and a system bus 1205 that couples the system memory 1204 to the central processing unit 1201. The computer device 1200 also includes a basic input / output system (Input / Output, I / O) system 1206 that helps transfer information between the various devices within the computer, and a mass storage device 1207 for storing an operating system 1213, application programs 1214, and other program modules 1215.

[0211] The basic input / output system 1206 includes a display 1208 for displaying information and an input device 1209 such as a mouse, keyboard, or the like for inputting information from a user. The display 1208 and the input device 1209 are both connected to the central processing unit 1201 through an input / output controller 1210 connected to the system bus 1205. The basic input / output system 1206 can also include an input / output controller 1210 for receiving and processing input from a keyboard, mouse, or electronic stylus, and other devices. Similarly, the input / output controller 1210 also provides output to a display screen, printer, or other types of output devices.

[0212] The mass storage device 1207 is connected to the central processing unit 1201 through a mass storage controller (not shown) connected to the system bus 1205. The mass storage device 1207 and its associated computer-readable media provide non-volatile storage for the computer device 1200. That is, the mass storage device 1207 can include a computer-readable medium (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.

[0213] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid state memory technology, CD-ROM, digital versatile disks (DVD), or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Computer storage media would not, however, include communication media including wired or wireless signaling media that transmit program code in the form of a carrier wave. The system memory 1204 and mass storage device(s) 1207 described above can collectively be referred to as memory.

[0214] According to various embodiments of the present application, the computer device 1200 can also operate in a networking environment via the network 1212, which is connected to the computer device 1200 through a network interface 1211 connected to the system bus 1205. It can also be appreciated that the network interface 1211 can also be utilized to connect to other types of networks or remote computer systems (not shown).

[0215] The memory further includes a computer program stored therein and configured to be executed by one or more processors to implement the above-mentioned identity recognition method, or to implement the above-mentioned training method of the identity recognition model.

[0216] In the exemplary embodiments, a computer-readable storage medium is also provided, in which at least one instruction, at least one program, a code set or an instruction set is stored, and when executed by a processor, the at least one instruction, the at least one program, the code set or the instruction set is used to implement the above-mentioned identity recognition method, or to implement the above-mentioned training method of the identity recognition model.

[0217] Optionally, the computer readable storage medium can include: ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State Drives), optical disc, etc. Among them, the random access memory can include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0218] In the example embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned identity recognition method, or executes the training method of the above-mentioned identity recognition model.

[0219] It should be understood that "multiple" mentioned herein refers to two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship. In addition, the step numbers described herein only exemplarily show a possible execution order between steps. In some other embodiments, the above steps can also be executed in a different order from the number order, such as executing two different numbered steps at the same time, or executing two different numbered steps in an order opposite to the illustration, and the embodiments of the present application are not limited to this.

[0220] The above only describes the example embodiments of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An identity recognition method, characterized by, The method comprises: acquiring a plurality of images containing an object to be identified; respectively performing feature extraction processing on the plurality of images to obtain feature information of each of the images; performing confidence evaluation processing in a hypersphere space based on a feature map and a feature vector of the image by a confidence evaluation network to obtain a confidence corresponding to the image; wherein the confidence is used to indicate a reliable degree of the image in an identity identification process for the object to be identified, the feature map is a feature image containing spatial structure information of the object to be identified obtained by using a feature extraction network, and the feature vector is a vector representation obtained by performing dimension reduction processing on the feature map of the image by using a feature mapping network; performing fusion processing on the feature information of the plurality of images according to the confidence corresponding to each of the plurality of images to obtain fusion feature information; determining the identity of the object to be identified according to the fusion feature information.

2. The method of claim 1, wherein, The fusion feature information comprises a fusion feature vector. The fusion processing on the feature information of the plurality of images according to the confidence corresponding to each of the plurality of images to obtain the fusion feature information comprises: performing fusion processing on the feature map of the plurality of images according to the confidence corresponding to each of the plurality of images to obtain a fusion feature map; and mapping the fusion feature map into a fusion feature vector; wherein the feature map is a feature image containing spatial structure information of the object to be identified obtained by using a feature extraction network. Or, performing fusion processing on the feature vector of the plurality of images according to the confidence corresponding to each of the plurality of images to obtain a fusion feature vector; wherein the feature vector is a vector representation obtained by performing dimension reduction processing on the feature map of the image by using a feature mapping network, and the feature map is a feature image containing spatial structure information of the object to be identified obtained by using a feature extraction network.

3. The method of claim 1, wherein, The fusion processing on the feature information of the plurality of images according to the confidence corresponding to each of the plurality of images to obtain the fusion feature information comprises: determining a fusion weight corresponding to each of the plurality of images based on the confidence corresponding to each of the plurality of images; performing weighted summation processing on the feature information of the plurality of images based on the fusion weight corresponding to each of the plurality of images to obtain the fusion feature information.

4. The method according to any one of claims 1 to 3, characterized in that, The method is implemented by an identity identification model, and the identity identification model comprises a feature extraction network, a feature mapping network, a confidence evaluation network, a feature fusion unit, and an identity identification unit; wherein the feature extraction network is configured to perform feature extraction processing on the plurality of images to obtain a feature map of each of the images; the feature mapping network is configured to map the feature map of the image into a feature vector of the image; the confidence evaluation network is configured to determine a confidence corresponding to the image based on the feature map and the feature vector of the image; the feature fusion unit is configured to perform fusion processing on feature information of the plurality of images according to the confidence corresponding to each of the plurality of images to obtain the fusion feature information; wherein the feature information comprises the feature map and / or the feature vector. The identity recognition unit is configured to determine the identity of the to-be-recognized object according to the fused feature information.

5. A method for training an identity recognition model, the method comprising: The identity recognition model comprises a feature extraction network, a feature mapping network and a confidence evaluation network, and the method comprises: obtaining a training sample set, the training sample set comprising a sample image set of at least one identity, and each sample image set comprising a plurality of sample images; performing feature extraction processing on the sample images based on the feature extraction network and the feature mapping network after training to obtain feature information of the sample images; obtaining center feature information of the identity based on the feature information of the plurality of sample images of the identity; training the confidence evaluation network using the center feature information of the identity and the feature information of the plurality of sample images of the identity, wherein the confidence evaluation network is configured to determine a confidence corresponding to each image of a to-be-recognized object based on feature information of the plurality of images, and the confidence corresponding to each image is used to fuse the feature information of each image to determine the identity of the to-be-recognized object; generating the identity recognition model comprising the confidence evaluation network.

6. The method of claim 5, wherein, The training of the confidence evaluation network using the center feature information of the identity and the feature information of the plurality of sample images of the identity comprises: performing confidence evaluation processing on the feature information of the sample images of the identity based on the confidence evaluation network to obtain the confidence of the sample images of the identity; determining a loss function of the confidence evaluation network based on the center feature information of the identity, the feature information of the plurality of sample images of the identity and the confidence of the plurality of sample images of the identity; adjusting parameters of the confidence evaluation network based on the loss function.

7. The method of claim 5, wherein, The obtaining of the center feature information of the identity based on the feature information of the plurality of sample images of the identity comprises: selecting a plurality of standard images from the plurality of sample images of the identity, performing feature extraction processing on the plurality of standard images of the identity based on the feature extraction network and the feature mapping network after training to obtain feature information of the plurality of standard images, and performing average processing on the feature information of the plurality of standard images to obtain the center feature information of the identity; or performing feature extraction processing on the plurality of sample images of the identity based on the feature extraction network and the feature mapping network after training to obtain feature information of the plurality of sample images, determining classification weights of the feature information of the plurality of sample images, selecting feature information with the largest classification weight as the center feature information of the identity, wherein the classification weight is used to indicate the closeness between the feature information of the sample images and the identity, and the classification weight is directly proportional to the closeness.

8. The method of claim 5, wherein, The feature extraction processing on the sample images based on the feature extraction network and the feature mapping network after training to obtain the feature information of the sample images comprises: The feature extraction network is trained, and feature extraction is performed on the sample image to obtain a feature map of the sample image, the feature map including spatial structure information of a sample object included in the sample image. The feature map of the sample image is mapped to a feature vector of the sample image by using the trained feature mapping network. The feature information includes the feature map of the sample image and the feature vector of the sample image.

9. The method according to any one of claims 5 to 8, characterized in that, The identity recognition model further includes a feature fusion unit and an identity recognition unit, wherein The feature fusion unit is configured to fuse feature information of the multiple images according to the confidence degrees of the multiple images respectively, to obtain fused feature information, wherein the feature information includes a feature map and / or a feature vector. The identity recognition unit is configured to determine an identity of a to-be-recognized object included in the multiple images according to the fused feature information.

10. An identity recognition apparatus, characterized by comprising: The apparatus includes: An image acquisition module configured to acquire multiple images including a to-be-recognized object; A feature extraction module configured to perform feature extraction on the multiple images respectively to obtain feature information of each of the images; A confidence degree determination module configured to perform confidence degree evaluation in a hyperspherical space based on a feature map and a feature vector of the image by using a confidence degree evaluation network, to obtain a confidence degree corresponding to the image, wherein the confidence degree is used to indicate a reliability of the image in an identity recognition process of the to-be-recognized object, the feature map is a feature image including spatial structure information of the to-be-recognized object obtained by using a feature extraction network, and the feature vector is a vector representation obtained by performing dimension reduction on the feature map of the image by using a feature mapping network; A feature fusion module configured to fuse feature information of the multiple images according to the confidence degrees of the multiple images respectively, to obtain fused feature information; An identity recognition module configured to determine an identity of the to-be-recognized object according to the fused feature information. 11.A device for training an identity recognition model, comprising: The identity recognition model includes a feature extraction network, a feature mapping network, and a confidence degree evaluation network, and the apparatus includes: A sample acquisition module configured to acquire a training sample set, the training sample set including sample image sets of at least one identity, and each sample image set of an identity including multiple sample images; A feature acquisition module configured to perform feature extraction on the sample images based on the trained feature extraction network and the feature mapping network, to obtain feature information of the sample images; An information acquisition module configured to obtain center feature information of the identity based on the feature information of the multiple sample images of the identity; A network training module configured to train the confidence degree evaluation network by using the center feature information of the identity and the feature information of the multiple sample images of the identity, wherein the confidence degree evaluation network is configured to determine a confidence degree corresponding to each of multiple images of a to-be-recognized object based on feature information of the multiple images, and the confidence degree corresponding to each of the images is used to fuse feature information of each of the images to determine an identity of the to-be-recognized object. A model generation module configured to generate the identity recognition model comprising the confidence evaluation network.

12. A computer device, comprising: The computer device comprises a processor and a memory, and the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the identity recognition method according to any one of claims 1 to 4, or to implement the identity recognition model training method according to any one of claims 5 to 9.

13. A computer-readable storage medium, characterized in that, The storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the identity recognition method according to any one of claims 1 to 4, or to implement the identity recognition model training method according to any one of claims 5 to 9.

14. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is loaded and executed by the processor to implement the identity recognition method according to any one of claims 1 to 4, or to implement the identity recognition model training method according to any one of claims 5 to 9.

Images