Method for determining identification results, device for determining identification results, computer device, and computer program

Abstract

A method, device, apparatus, and storage medium for determining an identification result relate to the technical field of artificial intelligence. The method includes the steps of: acquiring N types of images obtained by collecting images of a target to be identified; determining a weighting factor for an i-th type among the N types based on the imaging quality of the i-th type image; obtaining an identification result for each candidate user account among a plurality of candidate user accounts based on the weighting factors for the N types and the image match degrees of the N types; and determining a user account that matches the target to be identified based on each identification result of the plurality of candidate user accounts, wherein the image match degree of the i-th type represents the feature match degree between the target to be identified and the candidate user account for the i-th type image. The imaging quality of a certain type of image controls the contribution of the certain type of image in the identification result process, thereby improving the accuracy of target identification.

Description

[Technical Field]

[0001] This application claims priority from a Chinese patent application filed on April 12, 2023, application number 202310415584.8, entitled "Method, device, apparatus and storage medium for determining identification results," the entire contents of which are incorporated herein by reference.

[0002] The present embodiment relates to the technical field of artificial intelligence, and in particular to a method, a device, an apparatus and a storage medium for determining a recognition result. [Background technology]

[0003] With the development of artificial intelligence technology, target identification and authentication technology is constantly being updated. The application scenarios of target identification and authentication technology have expanded to include palm access restriction release, palm authentication payment, palm authentication time card punching, facial recognition payment, and facial recognition time card punching, etc. Target identification and authentication technology has brought more convenience to users' lives.

[0004] In the related technology, a related device collects full-color and infrared images of a target to be identified, and identifies a user account corresponding to the palm of the hand based on the full-color and infrared images. Summary of the Invention [Problem to be solved by the invention]

[0005] However, due to the complex shooting environment in the offline use process of related equipment, the image quality of images obtained using different collection methods will be inconsistent, which will affect the accuracy of matching identification based on different types of images. [Means for solving the problem]

[0006] The present embodiment provides a method, device, apparatus and storage medium for determining a target identification result, which can improve the accuracy of target identification. The technical means are as follows:

[0007] A method for determining an identification result provided according to one aspect of an embodiment of the present application is executed by a computer device and includes the steps of: acquiring N types of images (N is an integer greater than 1) obtained by collecting images of a target to be identified; for an i-th type (i is a positive integer equal to or less than N) among the N types, determining a weighting coefficient for the i-th type based on the imaging quality of the i-th type image; for each of a plurality of candidate user accounts, obtaining an identification result for the candidate user account based on the weighting coefficients for the N types and the image match degrees of the N types; and determining a user account that matches the target to be identified from the plurality of candidate user accounts based on the identification result for each of the plurality of candidate user accounts, wherein the different types of images are obtained using different collection methods, and the image match degree of the i-th type is used to represent the feature match degree between the target to be identified and the candidate user account regarding the i-th type of image.

[0008] An apparatus for determining an identification result provided according to one aspect of an embodiment of the present application includes: a multiple-type image acquisition module for acquiring N types of images (N is an integer greater than 1) obtained by collecting images of a target to be identified; a weighting coefficient acquisition module for determining a weighting coefficient of the i-th type (i is a positive integer less than or equal to N) of the N types based on the imaging quality of the i-th type image; an identification result acquisition module for acquiring an identification result of each candidate user account among a plurality of candidate user accounts based on the weighting coefficients of the N types and the image match degrees of the N types; and a user account matching module for determining a user account that matches the target to be identified from the plurality of candidate user accounts based on the identification results of each of the plurality of candidate user accounts, wherein the different types of images are obtained using different collection methods, and the image match degree of the i-th type represents the feature match degree between the target to be identified and the candidate user account regarding the i-th type of image.

[0009] A computer device provided according to one aspect of the present embodiment includes a processor and a memory, and a computer program is stored in the memory. The computer program is loaded and executed by the processor to realize the method for determining the identification result.

[0010] According to one aspect of the present invention, a computer-readable storage medium stores a computer program, and the method for determining the identification result is realized by loading and executing the computer program by a processor.

[0011] According to one aspect of the present invention, there is provided a computer program product, comprising: a computer program stored in a computer-readable storage medium, the computer program being read by a processor of a computer device from the computer-readable storage medium and executed by the processor to cause the computer device to perform the method for determining an identification result. [Effects of the Invention]

[0012] The technical solution provided in the present embodiment may have the following beneficial effects: after acquiring multiple different types of images corresponding to the target to be identified, a weighting factor for each type of image is determined based on the imaging quality corresponding to each of the multiple different types of images, and then the images are combined based on the weighting factor for each type of image to determine a user account that matches the target to be identified from multiple candidate user accounts. In this way, the present embodiment rationally and accurately assigns the weighting factor for each type of image based on the imaging quality of the images, thereby contributing to improving the accuracy of the identification result and ultimately improving the accuracy of target identification. In particular, in situations with complex lighting, rationally and accurately assigning the weighting factor for each type of image can increase the impact of images with good imaging quality on the identification result and decrease the impact of images with poor imaging quality on the identification result, thereby improving the accuracy of target identification. [Brief explanation of the drawings]

[0013] In order to more clearly describe the technical means in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will now be briefly described.

[0014] [Figure 1] FIG. 1 is a schematic diagram of the implementation environment of the technical means provided in one embodiment of the present application. [Figure 2] FIG. 2 is a schematic diagram of a quality assessment model provided in one embodiment of the present application. [Figure 3] FIG. 3 is a flowchart of a method for determining an identification result provided in one embodiment of the present application. [Figure 4] FIG. 4 is a flowchart of a method for determining an identification result provided in another embodiment of the present application. [Figure 5] FIG. 5 is a schematic diagram of a method for obtaining a bounding box provided in one embodiment of the present application. [Figure 6] FIG. 6 is a schematic diagram of a target detection method provided in one embodiment of the present application. [Figure 7] FIG. 7 is a schematic diagram of a target detection network provided in one embodiment of the present application. [Figure 8] FIG. 8 is a schematic diagram of key points provided in one embodiment of the present application. [Figure 9] FIG. 9 is a schematic diagram of a keypoint detection network provided in one embodiment of the present application. [Figure 10] FIG. 10 is a schematic diagram of a method for obtaining a region of interest map provided in one embodiment of the present application. [Figure 11] FIG. 11 is a schematic diagram of a region of interest diagram provided in one embodiment of the present application. [Figure 12] FIG. 12 is a schematic diagram of a quality evaluation network provided in one embodiment of the present application. [Figure 13] FIG. 13 is a flowchart of a method for obtaining weighting coefficients provided in an embodiment of the present application. [Figure 14] FIG. 14 is a schematic diagram of a palm authentication method provided in one embodiment of the present application. [Figure 15] FIG. 15 is a schematic diagram of a method for obtaining the number of discrimination points provided in one embodiment of the present application. [Figure 16] FIG. 16 is a block diagram of an apparatus for determining an identification result provided in one embodiment of the present application. [Figure 17] FIG. 17 is a block diagram of an apparatus for determining an identification result provided in another embodiment of the present application. [Figure 18] FIG. 18 is a block diagram of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0015] In order to make the objectives, technical means and advantages of the present application clearer, the following embodiments of the present application will be described in more detail in combination with the drawings.

[0016] AI (Artificial Intelligence) is the theory, method, technology, and application system that uses digital computers or devices controlled by digital computers to imitate, extend, and expand human intelligence to sense the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology of computer science that aims to understand the essence of intelligence and create new intelligent devices that respond in a manner similar to human intelligence. AI also involves researching the design principles and implementation methods of various intelligent devices to give them sensing, reasoning, and decision-making capabilities.

[0017] Artificial intelligence technology is a comprehensive academic field that spans a wide range of fields, including both hardware and software. The basic technologies of artificial intelligence generally include sensors, AI-specific chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. AI software technology primarily encompasses several major fields, including computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0018] Computer vision (CV) is the science of how machines "see." More specifically, it refers to the use of cameras and computers to perform machine vision tasks such as identifying, tracking, and measuring targets, and then processing the images to create images suitable for human observation or for transmission to instruments for detection. As a scientific discipline, CV studies related theories and technologies to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image identification, image semantic understanding, image retrieval, video processing, OCR (Optical Character Recognition), video semantic understanding, video content / action recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, and map building. It also includes common feature recognition technologies such as facial recognition and fingerprint recognition.

[0019] ML (Machine Learning) is an interdisciplinary field that spans multiple disciplines, including probability theory, statistics, approximation theory, convex analysis, and computational complexity theory. It specializes in the study of how computers imitate or realize human learning behavior, acquire new knowledge or skills, and reorganize existing knowledge structures to continually improve their performance. Machine learning is the core of artificial intelligence and the fundamental means of endowing computers with intelligence, and is applied across various fields of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, Bayesian networks, reinforcement learning, transfer learning, inductive learning, and imitation learning.

[0020] The technical means provided in the embodiments of the present application relate to artificial intelligence computer vision technology and machine learning technology, and use computer vision technology to process images to obtain image quality scores and target features of targets to be identified in the images, and then identify the targets to be identified based on the image quality scores and the target features of the targets to be identified to obtain identification results.In addition, the embodiments of the present application further use machine learning technology to train a quality assessment model based on the image quality scores, and obtain a quality assessment model that can accurately obtain image quality scores.

[0021] The technical means provided in the embodiments of the present application are applicable to any scenario requiring target identification, such as palm recognition scenarios, face recognition scenarios, target identification scenarios, security inspection scenarios, payment scenarios, etc. The technical means provided in the embodiments of the present application can improve the accuracy of target identification. The embodiments of the present application are not limited to the target to be identified. It can refer to any single object, such as a human being or a part of a human being.

[0022] FIG. 1 shows a schematic diagram of the implementation environment of the technical means provided in one embodiment of the present application. The implementation environment includes a terminal device 10 and a server 20.

[0023] The terminal device 10 may be, for example, a mobile phone, a tablet computer, a multimedia player, a PC (Personal Computer), a smart robot, an in-vehicle terminal, an access control device, a payment device, a safety inspection device, or any electronic device with an image capture function. A target application client is installed on the terminal device 10. The target application may be, for example, a palm recognition type application, a payment type application, a social entertainment type application, a simulation learning type application, or any application with a target identification function.

[0024] The server 20 is used to provide background services to clients of a target application (e.g., a palm authentication type application) in the terminal device 10. For example, the server 20 may be a background server of the application (e.g., a palm authentication type application). The server 20 may be a single server, a cluster server consisting of multiple servers, or a cloud computing server center.

[0025] The terminal device 10 and the server 20 can communicate with each other via a network 30. The network 30 may be a wired network or a wireless network.

[0026] For example, the terminal device 10 acquires N types of images (also referred to as multiple types of images) of a target to be identified, and then transmits the N types of images to the server 20. For each type (referred to as the i-th type) among the N types, the server 20 determines a weighting factor for the i-th type based on the imaging quality of the i-th type image. The server 20 performs feature identification on the i-th type image using the feature extraction model 50 to identify target features of the i-th type, and performs a matching operation between the target features of the i-th type and candidate features of the i-th type to obtain an image match degree for the i-th type. The server 20 then calculates a weighted sum of the image match degrees for the N types based on the weighting factors for the N types to obtain an identification result for the candidate user accounts. The server 20 then determines user accounts that match the target to be identified (i.e., a final identification result) based on each identification result. When the server 20 transmits the final identification result to the terminal device 10, the terminal device 10 performs an appropriate operation based on the final identification result.

[0027] Optionally, the quality assessment model 40 and the feature extraction model 50 may be located in the terminal device 10, so that the process of obtaining the final identification result is executed by the terminal device 10. However, the embodiment of the present application is not limited thereto.

[0028] The quality assessment model 40 is a neural network model for evaluating the imaging quality of an image and can output a quality score for evaluating the imaging quality of the image. The feature extraction model 50 is a neural network model for performing feature extraction on an image and can output target features for representing targets to be identified in the image. The target features, also referred to as feature vectors, are mathematical abstract representations of targets and are used to identify the targets. Optionally, the feature extraction model 50 can be constructed based on, for example, convolutional neural networks (CNNs), backpropagation (BP) neural networks, GoogleNet (a type of deep learning architecture), or residual neural networks (ResNets).

[0029] 2, the quality assessment model 40 includes a target detection network 410, a keypoint detection network 420, and a quality assessment network 430. The target detection network 410 is used to obtain a bounding box of a target to be identified in an image. The keypoint detection network 420 is used to obtain keypoints corresponding to the target to be identified. The quality assessment network 430 is used to obtain the imaging quality of the target to be identified in the image. For example, after the target detection network 410 obtains the bounding box of the target to be identified in the image, the image is cropped using the bounding box to obtain a cropped image, and the keypoint monitoring network 420 processes the cropped image.

[0030] Optionally, the target detection network 410 may be based on SSD (Single Shot Detector, a type of target detection network) or YOLO (You Only Look Once, a type of target detection network). It may also be based on other networks, such as CNN (Convolutional Neural Network), R-CNN (Region-CNN, a region-based convolutional neural network), or Faster R-CNN. This is not a limitation of the present embodiment. The target detection network 410 may be trained based on training samples containing targets to be identified. The training samples include photos of the targets to be identified and label data for the targets to be identified. The training data for the models to be identified may be determined based on the actual application of the target detection network in the operational process, and this is not a limitation of the present embodiment.

[0031] The keypoint detection network 420 can be constructed based on networks such as DeepPose (a type of convolutional neural network for keypoint detection), DNN (Deep Neural Network), HRNet (High-Resolution Net), etc., but the embodiments of the present application are not limited thereto. The keypoint detection network 420 is trained based on training samples to which keypoint label data is added.

[0032] The quality assessment network 430 may be constructed based on a network such as FaceQnet (a type of quality assessment model for facial images), CNN, or GoogleNet. The quality assessment network 430 may be trained based on training samples to which label data of imaging quality is assigned, or may be trained using the discrimination scores output by FaceQnet as label data. However, the present embodiment is not limited to this.

[0033] Next, the technical means provided in this application will be explained by way of method examples.

[0034] Figure 3 shows a flowchart of a method for determining an identification result provided in one embodiment of the present application. Each step of the method is performed by a terminal device 10 or a server 20 in the implementation environment of the technical solution shown in Figure 1. For convenience of explanation, the technical solution will be described below using a computer device as the execution entity. The method includes the following steps (310 to 340):

[0035] In step 310, N types of images are acquired by performing acquisition on the target to be identified, where the different types of images are acquired using different acquisition methods, and N is an integer greater than 1.

[0036] In some embodiments, the target to be identified refers to the object to be identified in the target identification process. By determining the identification result of the target to be identified, the type of the target to be identified can be determined or the identity authentication of the target to be identified can be completed. In the embodiments of this application, the type of the target to be identified is not limited. For example, in a palm recognition scenario, the target to be identified refers to the palm. Also, for example, in a face recognition payment scenario, the target to be identified refers to the face. The specific value of N can be determined according to actual identification needs and is not limited here.

[0037] In some embodiments, each type of image among the N types of images has an image to be identified displayed therein, and the N types of images include at least two of an infrared image, a full-color image, a depth image, a grayscale image, etc. Optionally, there are at least two types of images among the N types of images, and the at least two types of images are obtained by respectively acquiring images of the target object using different acquisition methods. An acquisition method may be understood as a method for capturing an image of the target to be identified.

[0038] Optionally, for an i-th type of the N types of images, the i-th type of image includes k images, where k is a positive integer. The k images are acquired in the same manner. For different types of the N types, the number of images included in each of the different types may be the same or different.

[0039] In some embodiments, the different acquisition schemes are achieved by different acquisition sensors. Optionally, the N types of images are obtained by performing image collection of a target to be identified by an image collection device. For example, N types of collection sensors are arranged in the image collection device, and an i-th type collection sensor among the N types of collection sensors collects an image of the target to be identified using a corresponding collection method to obtain an i-th type image, where i is a positive integer equal to or less than N.

[0040] For example, the maximum time interval between the collection timings of the different types of collection sensors is equal to or less than the interval threshold. For example, if the interval threshold is 0.5 seconds and the sensor with the earliest collection timing among the N types of collection sensors collects an image requiring identification at 12:00:00.00, the time at which the other collection sensors among the N types of collection sensors collect an image requiring identification for a target requiring identification will not be later than 12:00:00.05.

[0041] Optionally, an i-th type collection sensor among the N types of collection sensors collects images of the target to be identified multiple times, and transmits the top k images with the best focus from the images obtained in each of the multiple collections to the computer device as the i-th type images, where k is a positive integer. For example, k is equal to 3.

[0042] For example, taking a palm authentication scene as an example, the target to be identified is the palm of the hand to be identified, and the image collection device may be called a palm scanning device. In this scene, if the palm scanning device is equipped with at least an infrared sensor and a color sensor, the N types of images include full-color images and infrared images. The infrared sensor in the palm scanning device obtains an infrared image by imaging with infrared light emitted from the palm of the hand to be identified. The color sensor in the palm scanning device obtains a full-color image by imaging with natural light reflected from the palm of the hand to be identified. Among these, the infrared image can also be used for in-vivo detection. For example, the infrared image also includes the distribution of blood vessels. The full-color image is also called an RGB (Red-Green-Blue) image. The full-color image is used to represent the distribution of the palm print on the palm of the hand to be identified.

[0043] In some embodiments, the computer device and the image collecting device are connected by wire or wirelessly. For example, if the computer device is a server, the image collecting device can wirelessly transmit the N types of images to the server via long-distance wireless communication, thereby achieving step 310. For example, if the image collecting device and the computer device are integrated into one device, the image collecting device can collect images of the target to be identified, and then the computer device can directly obtain the N types of images.

[0044] In step 320, for the i-th type among the N types, a weighting coefficient for the i-th type is determined based on the imaging quality of the image of the i-th type, where i is a positive integer equal to or less than N.

[0045] The i-th type may be any one of the N types. For each of the N types, a weighting factor for that type may be determined using the method provided in step 320. Optionally, the imaging quality of the i-th type image is used to determine the sharpness of k images included in the i-th type image. Higher sharpness of the k images indicates higher imaging quality of the i-th type image. Optionally, lower sharpness of the k images indicates lower imaging quality of the i-th type image. In other words, the imaging quality of the i-th type image reflects the richness of details related to the target to be identified included in the i-th type image, and higher imaging quality of the i-th type image indicates that more details related to the target to be identified are recorded in the i-th type image.

[0046] Optionally, the imaging quality of the i-th type of image is related to factors such as the degree of focus fixation during the process of collecting the target to be identified using the i-th collection method, the movement of the target to be identified, the image collection performance of the i-th collection sensor (e.g., resolution, aperture size), and the stability of the i-th collection sensor (whether or not blurring occurs). Since the imaging quality of an image has an important impact on the identification accuracy of the target to be identified, an image with high imaging quality contributes to improving the identification accuracy of identifying the target, but an image with high imaging quality can provide relatively little useful information in the target identification process. Therefore, before performing target identification, it is important to determine the contribution of different types of images to the identification process of the target to be identified based on the imaging quality of each of the N types of images.

[0047] Optionally, the imaging quality of the i-th type of image is obtained by predicting, based on a quality assessment model, at least one image included in the i-th type of image. For example, at least one image included in the i-th type of image is input into the quality assessment model, and the imaging quality of the at least one image is evaluated by the quality assessment model. For specific details regarding this step, please refer to the following examples.

[0048] Optionally, the imaging quality of the i-th type of image may be determined based on the smoothness between adjacent pixels in at least one image included in the i-th type of image. A larger smoothness value indicates a higher imaging quality of the image. A smaller smoothness value indicates a lower imaging quality of the image. Illustratively, for any one of the at least one image, a pixel value difference between two adjacent pixels in the positive direction of the x-axis of the image's screen coordinate system is identified, and pixel value differences whose numerical value is equal to or greater than a pixel threshold are added together to obtain a smooth value, which is used to represent the image quality of the image.

[0049] In this embodiment, the weighting coefficients are used to indicate the contribution of an image to the process of identifying a target to be identified. That is, the weighting coefficient of a certain type determines the contribution of an image of a certain type to the identification result. There are at least two types among the N types, and the weighting coefficients of the two types are different. Optionally, the sum of the weighting coefficients of the N types is equal to 1.

[0050] Optionally, for an i-th type among the N types, the value of the weighting factor of the i-th type is positively correlated with the imaging quality of the i-th type image, i.e., the higher the imaging quality of the i-th type image, the larger the value of the weighting factor of the i-th type, and the lower the imaging quality of the i-th type image, the smaller the value of the weighting factor of the i-th type.

[0051] In step 330, for each candidate user account among the plurality of candidate user accounts, an identification result of the candidate user account is obtained based on the N types of weight coefficients and the N types of image match degrees, where the i-th type of image match degree represents the feature match degree between the target to be identified and the candidate user account regarding the i-th type of image.

[0052] The target to be identified originates from a user belonging to one candidate user account among multiple candidate user accounts. That is, the purpose of identifying the target to be identified is to find the candidate user account among the multiple candidate user accounts based on the target to be identified. Optionally, the candidate user account refers to a user account that has completed registration with the target application, and one user account corresponds to one user. In step 330, the computer device determines the identification results between the target to be identified and each of the multiple candidate user accounts, and then determines the user account corresponding to the user to which the target to be identified belongs based on the identification results. For specific details, please refer to the examples described below.

[0053] Optionally, the image match degree of the i-th type is used to determine the similarity between the target to be identified and the image of the i-th type possessed by each of the candidate user accounts.

[0054] Optionally, for each user account among the candidate user accounts, determining the degree of image match of the ith type between the candidate user account and the target to be identified involves using information for representing the ith type image of the target to be identified and information for representing the ith type image of the candidate user account.

[0055] Illustratively, an image of the i-th type of target to be identified can be obtained in step 310 above, and information for representing the image of the i-th type of target to be identified can be obtained by processing the image of the i-th type of target to be identified.

[0056] For example, the i-th type image of the candidate user account is obtained by collecting data on a user target provided by a user belonging to the candidate user account. The i-th type image of the candidate user account may be uploaded to a computer device in advance from the candidate user account, and the computer device may process the i-th type image of the candidate user account to obtain information for representing the i-th type image of the candidate user account. The information for representing the i-th type image of the candidate user account may be determined in advance and stored in a database.

[0057] The user target provided by the user and the target to be identified belong to the same type. For example, the target to be identified and the user target provided by the user are both palms. That is, the user target is the palm of the user's hand. Also, for example, the target to be identified and the user target provided by the user are both pupils.

[0058] In some embodiments, after determining the N types of image match degrees between the candidate user account and the target to be identified, the computer device determines an identification result for the candidate user account based on the N types of image match degrees and the weight coefficients for each of the N types. The identification result for the candidate user account may be expressed as a decimal number in the range [0, 1]. Optionally, in the process of determining the identification result for a candidate user account, for an i-th type among the N types, the image match degree for the i-th type is processed using the i-th type weight coefficient to obtain a processed i-th type image match degree. The processed i-th type image match degree is used in the process of determining the identification result for the candidate user. For specific details regarding this step, please refer to the examples described below.

[0059] The imaging coefficient of the i-th type is related to the imaging quality of the i-th type image, and the two are positively correlated. Therefore, by increasing the weighting coefficient of the type with good imaging quality, the contribution of the image of that type to the confirmation process of the classification result can be increased, and the detailed information provided by the image of the type with good imaging quality can be fully utilized in the target classification process. At the same time, the interference of meaningless information in the image of the type with low imaging quality on the classification result can be reduced, reducing the introduction of erroneous information into the target classification process and contributing to improving the accuracy of the classification result.

[0060] Once the identification results for each of the multiple candidate user accounts have been determined, execution of step 330 is completed. Note that the process of determining the identification results for each of the multiple candidate user accounts may be performed sequentially or in parallel. For example, the computer device may sequentially determine the identification results for each of the multiple candidate user accounts. Alternatively, for example, if the computing performance of the computer device allows, the computer device may launch multiple processes in step 330, each of which separately calculates the identification results for a different candidate user account. Such a method increases the execution speed of target identification and shortens the time required to determine the identification results for each of the multiple candidate user accounts, thereby contributing to reducing user waiting time.

[0061] In step 340, a user account that matches the target to be identified is determined from the plurality of candidate user accounts based on the identification result of each of the plurality of candidate user accounts.

[0062] In some embodiments, the computer device determines the candidate user account corresponding to the identification result with the largest numerical value from among the identification results of each of the plurality of candidate user accounts as the user account that matches the target to be identified, i.e., determines that the target to be identified is provided by a user belonging to the user account that matches the target to be identified.

[0063] Optionally, the step of the computer device determining a user account that matches the target to be identified from among the plurality of candidate user accounts based on the identification results of each of the plurality of candidate user accounts includes the computer device determining a user account that matches the target to be identified based on the identification results of each of the plurality of candidate user accounts and a judgment threshold, and if the identification result with the largest numerical value among the identification results of each of the plurality of candidate user accounts is equal to or greater than the judgment threshold, the candidate user account corresponding to the identification result with the largest numerical value is determined to be the user account that matches the target to be identified.

[0064] For example, if the identification results of each of the plurality of candidate user accounts are all less than the decision threshold, it is determined that there is no user account among the plurality of candidate user accounts that matches the target to be identified. The decision threshold is predetermined. For example, the decision threshold is equal to 0.95.

[0065] As described above, after collecting multiple different types of images corresponding to the target to be identified, a weighting factor for each type of image is determined based on the imaging quality corresponding to each of the multiple different types of images, and then the images are combined based on the weighting factor for each type of image to determine a user account that matches the target to be identified. In this way, the present embodiment rationally and accurately assigns the weighting factor for each type of image based on the imaging quality of the image, thereby improving the accuracy of the identification result and ultimately the accuracy of target identification. Particularly in situations with complex lighting, rationally and accurately assigning the weighting factor for each type of image can increase the impact of images with good imaging quality on the identification result and decrease the impact of images with poor imaging quality on the identification result, thereby improving the accuracy of target identification.

[0066] Next, a method for determining weighting coefficients for different types among the N types will be described with some examples.

[0067] In some embodiments, determining the i-th type weighting factor based on the imaging quality of the i-th type image in step 320 includes the following steps, which are performed by a computer device:

[0068] In step 321, a quality score of the i-th type image is determined. The quality score of the i-th type image is used to represent the imaging quality of the i-th type image.

[0069] In some embodiments, the quality score is obtained by evaluating the screen content of the i-th type image, and the quality score can be used to represent the clarity and imaging quality of the corresponding area of ​​the target to be identified in the i-th type image. Optionally, the quality score is used to evaluate the imaging quality of the target to be identified in the image. Optionally, a higher quality score indicates a higher imaging quality of the target to be identified in the image, i.e., the image of the target to be identified in the i-th type image is clearer and more details of the target to be identified are included in the i-th type image. A lower quality score indicates a lower imaging quality of the i-th type image, i.e., the image of the target to be identified in the image is blurred and the image may not display the details of the target to be identified. This contributes to improving the identification accuracy of the target to be identified.

[0070] The quality score of the i-th type image may be determined by a quality assessment model, or may be obtained by evaluating the pixel values ​​of the pixels in the image using a method such as a smoothed value. For specific details of the quality assessment model and the smoothed value, please refer to the above examples and will not be described again here.

[0071] Optionally, when the i-th type of images includes k images, where k is greater than 1, determining the quality score of the i-th type of image includes determining a quality score for each of the k images, and determining the quality score of the i-th type of image based on the quality scores for each of the k images. For any one image in the k images, the quality score of the image may be determined by a quality assessment model or a smoothed value. See above for details.

[0072] Illustratively, the computer device obtains a quality score for the i-th type of image by calculating a weighted sum of the quality scores of the k images. Illustratively, the computer device identifies m images from the k images with the highest quality scores based on the quality scores of each of the k images, and obtains a quality score for the i-th type of image by calculating a weighted sum of the quality scores of each of the m images, where m is a positive integer less than or equal to k.

[0073] In step 323, a weight characteristic parameter of the i-th type is determined based on the numerical relationship between the quality score of the i-th type image and the quality score threshold, and the weight characteristic parameter of the i-th type is used to control the contribution of the i-th type image to the classification result.

[0074] In some embodiments, the quality score threshold is a reference value used when determining the weight characteristic parameter of the i-th type based on the quality score of the i-th type image. Optionally, the quality score threshold of the i-th type and the weight coefficient of the i-th type are positively correlated, i.e., the larger the value of the quality score threshold of the i-th type, the larger the value of the weight coefficient of the i-th type, and the smaller the value of the quality score threshold of the i-th type, the smaller the value of the weight coefficient of the i-th type.

[0075] Optionally, there are at least two different types among the N types, and the quality score thresholds for each of the two types are different. Exemplarily, for an i-th type of the N types, the quality score threshold for the i-th type is determined based on the image collection capability of the i-th type collection sensor. Determining the quality score threshold for the i-th type in this manner helps to improve the adaptability of the weighting coefficients to collection sensors with different collection capabilities, improves the accuracy of determining the weighting coefficient values, and ultimately contributes to improving the accuracy of the target identification process.

[0076] Optionally, different types of images may share the same quality score thresholds, which may reduce the number of quality score thresholds and reduce the complexity of determining the weighting factors.

[0077] In one example, the quality score threshold is preset, and the quality score threshold for each of the N types is stored in the computer device. In another example, the quality score threshold is obtained in real time. If the computer device is a server, the server may receive images of the N types collected from multiple targets to be identified at the same time. The multiple targets to be identified may be collected by different image collection devices. If the quality score threshold for each of the N types set for each different image collection device is stored in advance, the server's storage pressure will be large.

[0078] In this situation, when the image collection device transmits the N types of images collected to the server, the collection performance parameters of the N types of collection sensors included in the image collection device may be sent to the server so that the server can determine the quality score thresholds for each of the N types based on the collection performance parameters of the collection sensors. This method helps reduce the pressure on the server to store the quality score thresholds. In addition, determining the quality score parameters for each type in real time based on the collection performance parameters of the collection sensors contributes to improving the adaptability of the target identification process to different scenes.

[0079] The feature identification algorithm used in the present embodiment is considered to have a certain degree of robustness to imaging quality. That is, after the imaging quality of the i-th type image and the imaging quality of the j-th type image both reach a certain level (i.e., the imaging quality is good), the difference between the imaging quality of the i-th type image and the imaging quality of the j-th type image does not have a significant impact on the identification result. Therefore, when the imaging quality of a certain type of image reaches a certain level (e.g., the quality score of the image of the type reaches or exceeds the quality score threshold), the influence of the imaging quality of the image of the type on the weighting coefficient of the type decreases, and the weighting coefficient of the type no longer changes.

[0080] Optionally, the quality score threshold may be set and adjusted based on experience. The process will be described using an example in which a first type and a second type of image among the N types share the same quality score threshold, where the first type of image is a full-color image and the second type of image is an infrared image. In the process of determining the quality score threshold, multiple sets of sample images are acquired from the same target using an embodied collection device. Each set of sample images includes one high-quality full-color image and one high-quality infrared image. A default score threshold is set, and steps 320 to 340 are performed to adjust the quality score threshold for each set with the goal of maximizing the classification results for each set of sample images, thereby achieving weighting coefficient adjustment. If the classification results for each set of sample images tend to stabilize, the current quality score threshold for each set of sample images is determined as the final quality score threshold for the sample images in that set. The final quality score thresholds for each set of sample images are averaged to obtain a quality score threshold shared by the first and second types.

[0081] Optionally, the computing device selects one of the quality score of the i-th type image and the quality score threshold as the weight characteristic parameter of the i-th type based on a numerical relationship between the quality score of the i-th type image and the quality score threshold of the i-th type. In one example, the quality score mentioned in step 321 may be obtained in steps 315 to 318 in the following embodiment.

[0082] In some embodiments, step 323 may be accomplished in one of the following ways:

[0083] Method 1: The smaller value of the quality score of the image of the i-th type and the quality score threshold is determined as the weight characteristic parameter of the i-th type.

[0084] For example, the process of obtaining the weight characteristic parameters of the i-th type is determined using the formula min(Q, A), where Q is the quality score of the i-th type image and A is the quality score threshold of the i-th type.

[0085] Method 2: The larger value of the quality score of the image of the i-th type and the quality score threshold is determined as the weight characteristic parameter of the i-th type.

[0086] For example, the weight characteristic parameter of the i-th type is determined using the formula max(Q, A), where Q is the quality score of the i-th type of image, and A is the quality score threshold. According to this method, by setting the quality score threshold as the weight characteristic parameter for a type with low imaging quality, it is possible to prevent the weight characteristic parameter of the type from being too low, thereby reducing the contribution of the image match degree of the type to the classification result, and to prevent the calculation process for determining the image match degree of the type from becoming meaningless.

[0087] Method 3: Determine the weighted characteristic parameter of the i-th type as the average value of the quality score of the i-th type image and the quality score threshold. In the process of determining the weighted characteristic parameter of the i-th type, the weighted coefficients of the quality score of the i-th type image and the quality score threshold may be equal or unequal. The present application is not limited thereto.

[0088] For example, the quality score threshold of the i-th type of the i-th type. By determining the average value of the quality score of the i-th type image and the quality score threshold as the weight characteristic parameter of the i-th type, it helps to appropriately shorten the quality score threshold determination process, and helps to allow multiple types of images to participate in the target identification process based on the contribution of each type of image to the identification result of the candidate user account according to imaging quality, thereby providing different types of reference information for the target identification process and contributing to ensuring the accuracy of the target identification process from multiple aspects.

[0089] In step 325, a weight coefficient is calculated based on the weight characteristic parameter of the i-th type to obtain the i-th type weight coefficient.

[0090] Optionally, the computer device uses a linear weight adjustment formula to perform weight coefficient calculation on the i-th type weight characteristic parameter to obtain the i-th type weight coefficient. The weight coefficient calculation refers to a process of determining the i-th type weight coefficient based on the i-th type weight characteristic. Illustratively, the weight coefficient calculation process requires the use of weight characteristic parameters of each of the N types.

[0091] In some embodiments, step 325 is implemented by the following steps: The following steps are performed by a computer device.

[0092] In step 325-a, the weight characteristic parameters of each of the N types are summed to obtain a sum result.

[0093] For example, the total result is calculated using the following formula:

[0094]

number

[0095] However, C i represents the weight characteristic parameter of the i-th type.

[0096] Optionally, in the process in which the computer device determines the weighting coefficients for the N types, step 325-a is executed fewer than N times. That is, in determining the weighting coefficients for the N types in step 325-a, a process of determining a weighting coefficient for the third type, which is the type whose weighting coefficient is determined earliest among the N types, is essential. In an actual execution process, the third type may be one type randomly selected from the N types.

[0097] For example, for the N types of types other than the third type, step 325-a does not necessarily need to be performed in the process of determining the weighting coefficients. For example, after the total result is determined in the process of determining the weighting coefficient of the third type, the computer device can store the value of the total result, and when determining the weighting coefficients of the other types, it can directly read the total result from the storage space instead of performing step 325-a.

[0098] In step 325-b, a ratio calculation is performed between the i-th type weight characteristic parameter and the sum result to obtain the i-th type weight coefficient.

[0099] Optionally, the i-th type weighting coefficient and the i-th type weight characteristic parameter exhibit a positive correlation, and the i-th type weighting coefficient and the total result exhibit a negative correlation.

[0100] As an example, the N types include a first type and a second type, and the first type of image is a full-color image and the second type of image is an infrared image, so the multiple different types of images include full-color images and infrared images. For example, if the quality score thresholds of the first type and the second type are equal, the weighting coefficient α of the first type is calculated using the following formula:

[0101]

number

[0102] However, Q r represents the quality score of the first type of image, and Q i represents the quality score of the second type image, and A is the quality score threshold. According to the above, if the sum of the weighting coefficients of the N types of images is 1, the weighting coefficient β of the second type is expressed as β=1-α, or as shown in the following equation:

[0103]

number

[0104] The definitions of the parameters in the formula are as described above and are not limited here.

[0105] This method determines a weight characteristic parameter for each of the multiple types based on the quality scores of the multiple types of images, and determines weight coefficients for the multiple types based on the weight characteristic parameters. This allows the imaging qualities of different types of images to be taken into account in the process of determining the weight coefficients, and types with higher imaging quality have a greater contribution to the process of determining the identification result. Images with higher imaging quality can provide the target identification process with more information about the target to be identified, so this embodiment of the present application contributes to improving the accuracy of the target identification process.

[0106] Furthermore, weight characteristic parameters corresponding to different types of images are determined based on the magnitude relationship between the quality score and the quality score threshold, and weight coefficients corresponding to different types of images are determined based on the weight characteristic parameters corresponding to different types of images. This makes the weight coefficients more consistent with the actual influence relationship between the quality score and the identification result, thereby increasing the rationality of the weight coefficient determination and ultimately improving the accuracy of target identification. At the same time, the technical means provided in this embodiment can dynamically adjust the weight coefficients based on image quality, making the technical means provided in this embodiment adaptable to various lighting environments and improving the practicability and anti-interference capabilities of the technical means provided in this embodiment.

[0107] Next, a method for determining the quality score of an i-th type image will be described according to several embodiments. Figure 4 is a flowchart of a method for determining a classification result provided in another exemplary embodiment of the present application. The execution body of this embodiment is a computer device. As shown in Figure 4, the method for determining a classification result can be realized by the following several steps:

[0108] In step 310, N types of images are acquired by collecting the target to be identified, where different types of images are acquired using different collection methods, and N is an integer greater than 1. For details of step 310, please refer to the above description.

[0109] After performing step 310, the computing device needs to respectively determine a weighting coefficient for each type of image based on the imaging quality of each type of image. Optionally, in this embodiment, a quality score for the i-th type of image is determined by a quality assessment model. The quality assessment model includes a target detection network, a keypoint detection network, and a quality assessment network. The computing device uses the quality assessment model to determine the imaging quality of each of the N types of images. Optionally, the same quality assessment model is used to determine the imaging quality of different types of images.

[0110] Optionally, different quality assessment models are used to determine the imaging qualities of different types of images. Illustratively, the different quality assessment models include models with different model structures and models with the same model structure but different model parameters. In the present embodiment, the principles of using different types of quality assessment models to determine the imaging qualities of different types of images are similar. The following describes the process by taking the case of determining the imaging quality of an i-th type of image as an example.

[0111] In step 315, the target detection network performs target detection on the i-th type image to determine a bounding box diagram of the image of the target to be identified in the i-th type image, where the bounding box diagram refers to the imaging area of ​​the target to be identified in the i-th type image.

[0112] Optionally, the bounding box diagram in the i-th type image is used to separate the display area of ​​the target to be identified from the display area of ​​other screen content in the i-th type image. Illustratively, the bounding box diagram in the i-th type image includes all screen content related to the target to be identified in the i-th type image. Illustratively, the bounding box diagram is a rectangular area in the i-th type image.

[0113] Optionally, the computer device performs target detection on the i-th type image using a target detection network to obtain a bounding box of the target to be identified in the i-th type image, and obtains a bounding box diagram by cutting out an area corresponding to the bounding box from the i-th type image.

[0114] Referring to FIG. 5, the execution process of step 316 will be described using a palm scanning scene as an example. For an i-th type image 501 among N types of images, the i-th type image 501 is first input to a target detection network 502. The target detection network 502 divides the i-th type image 501 into an S*S grid and performs boundary prediction on each of the S*S grids to obtain a prediction result for each of the S*S grids. The process of performing boundary prediction on any one grid includes performing B bounding box prediction on that grid to obtain prediction information for each of the B bounding boxes. Note that the prediction result for that grid includes prediction information for each of the B bounding boxes, where B is a positive integer greater than 1. The size of the frame area of ​​each of the B bounding boxes is different. Typically, the bounding box is a rectangular frame.

[0115] The prediction information of each of the B bounding boxes includes five predicted values ​​x, y, w, h, and a confidence level (i.e., confidence), where x and y represent the planar rectangular coordinates of the center point of the bounding box, and the range of x and y values ​​is adjusted to 0 to 1 by normalization. w and h represent the size of the bounding box (e.g., including the width and height of the bounding box), and the range of w and h values ​​is adjusted to 0 to 1 by normalization.

[0116] Optionally, normalization for x is achieved by dividing the original value of x by the width w of the bounding box to obtain the predicted value x, and normalization for y is achieved by dividing the original value of y by the height h of the bounding box to obtain the predicted value y.

[0117] The confidence level is used to represent the likelihood that the entire palm or part of the palm is displayed within the corresponding image region of the bounding box in the i-th type image.

[0118] For example, if each grid predicts the probability of C hypothetical categories, the prediction result output by target detection network 502 will be a tensor of S*S*(B*5+C). For example, referring to FIG. 6, if image 501 is divided into 7*7 grids (i.e., S=7), each grid corresponds to two bounding boxes (i.e., B=2), and there are a total of 20 hypothetical categories (i.e., C=20), the prediction result of target detection network 502 will be a tensor of 7*7*30.

[0119] Optionally, the confidence may be calculated as follows:

[0120]

number

[0121] where Pr indicates whether a palm exists in the corresponding grid (Pr=1 if it exists, Pr=0 if it does not exist).

number

[0122] Optionally, the target detection network 502 mainly uses GoogleNet to extract features. For example, referring to Figure 7, the target detection network 502 includes a convolutional layer and a fully connected layer. The convolutional layer is used to extract features from the image 501, and can sequentially obtain feature vectors of, for example, 448*448*3, 112*112*192, 56*56*256, 28*28*512, 14*14*1024, 7*7*1024, and 7*7*1024 dimensions. The fully connected layer is used to predict category, coordinate, size, confidence, etc. based on the features output by the convolutional layer, and can sequentially obtain feature vectors of, for example, 4096 dimensions and 7*7*30 dimensions.

[0123] Optionally, in a palm scanning scene, the target detection network 502 may only assume the palm as its hypothesized category, and determine the bounding box with the highest confidence, for example, the bounding box 503 in Figure 5, as the palm bounding box. The bounding box 503 is used to represent the position of the palm in the image 501. (x, y) is the pixel position of the upper left corner of the bounding box 503, and (w, h) is the width and height of the bounding box 503.

[0124] In step 316, keypoint detection is performed on the bounding box diagram using a keypoint detection network to determine at least one keypoint of the target to be identified in the bounding box diagram.

[0125] The target detection network can effectively capture the image content of the target to be identified, which helps to effectively prevent irrelevant content from affecting the identification of the target to be identified, thereby improving the accuracy of target identification. However, during the image acquisition process, if the position or angle between the target to be identified and the image acquisition device changes, distortion may occur in the image, and the size of the bounding box diagram at different distances may be inaccurate. Therefore, the embodiment of the present application further uses a keypoint detection network to obtain keypoints corresponding to the target to be identified, and further processing is performed on the bounding box diagram to prevent the sampling position, sampling angle, etc. from affecting the accuracy of target identification.

[0126] A key point refers to a point that is relatively fixed on the target to be identified and can be used to indicate the position of the target to be identified. For example, with reference to FIG. 8 , taking the palm of a hand as an example, the points between the fingers may be determined as at least one key point corresponding to the palm, i.e., key point 801, key point 802, key point 803, and key point 804.

[0127] 9, when a bounding box diagram 901 (e.g., having a dimension of 220*220) is input to a keypoint detection network 902, the keypoint detection network 902 performs keypoint detection on the palm of the hand in the bounding box diagram 901, thereby obtaining at least one keypoint corresponding to the palm. For example, the keypoint detection network 902 performs a series of convolution processes on the bounding box diagram 901, such as sequential convolution processes with dimensions of 22*55*48, 27*27*128, 13*13*192, and 13*13*192, and then performs position prediction for each keypoint using two fully connected layers (e.g., each having a dimension of 4096), thereby outputting predicted coordinates within the bounding box diagram 901 for at least one keypoint.

[0128] To solve the problem of the bounding box diagram being of a non-uniform size and the keypoint detection network 902 only accepting inputs of a preset size, it is necessary to scale bounding box diagrams that do not match the preset size. This can result in errors in the predicted keypoint coordinates due to the scaling of the bounding box diagram (e.g., if the bounding box scale is too large, it must be scaled to the preset size). This embodiment further extracts a cropped image of the preset size from the bounding box diagram based on the predicted keypoint coordinates, and then the keypoint detection network 902 performs keypoint detection on the cropped image to obtain the final predicted positions corresponding to the keypoints. This improves the accuracy of keypoint extraction and target identification.

[0129] In step 317, the bounding box map is cropped based on at least one keypoint to obtain a region of interest map of the target to be identified, which includes discriminative features for identifying the target to be identified.

[0130] The region of interest map refers to an image obtained by cutting out the image content of a corresponding region from a bounding box map based on a region of interest (ROI) corresponding to a target to be identified. For example, taking the palm of a hand as an example, the region of interest of the palm may refer to the area of ​​the palm that has a palmprint. In this case, the region of interest map of the palm is a cutout map corresponding to the palmprint area of ​​the bounding box map of the palm.

[0131] Optionally, the computing device identifies a minimum bounding rectangle that encloses at least one keypoint, and crops the bounding box diagram to a portion of the image outside the minimum bounding rectangle, the remaining portion being the region of interest diagram.

[0132] In step 318, a quality assessment is performed on the region of interest map by the quality assessment network to determine a quality score of the i-th type image, which is used to represent the imaging quality of the i-th type image.

[0133] Optionally, if the i-th type of image includes k images, where k is greater than 1, steps 315, 316, 317, and 318 are performed for each of the k images. After determining the imaging qualities of the k images, the imaging quality of the i-th type of image is determined according to the imaging qualities of the k images. For specific details of these steps, please refer to the above-mentioned embodiments, and they will not be described again here.

[0134] In step 320, for the i-th type among the N types, a weighting coefficient for the i-th type is determined based on the imaging quality of the image of the i-th type, where i is a positive integer equal to or less than N.

[0135] In step 330, for each candidate user account among the plurality of candidate user accounts, an identification result of the candidate user account is obtained based on the N types of weight coefficients and the N types of image match degrees, where the i-th type of image match degree represents the feature match degree between the target to be identified and the candidate user account regarding the i-th type of image.

[0136] In step 340, a user account that matches the target to be identified is determined from the plurality of candidate user accounts based on the identification result of each of the plurality of candidate user accounts.

[0137] Please refer to the above-mentioned embodiment for the specific contents of steps 320, 330, and 340, and they will not be described again here.

[0138] By using a quality assessment model to perform quality assessment on the region of interest map of the target to be identified, the imaging quality of the target to be identified can be accurately reflected.Furthermore, by training the quality assessment model based on the region of interest map, the quality assessment model can be prevented from being influenced by invalid data, thereby improving the accuracy of obtaining quality scores and ultimately improving the accuracy of target identification.

[0139] Next, a process for obtaining a region of interest map will be described in accordance with some embodiments. In some embodiments, step 317 can be realized by the following several sub-steps, which are performed by a computer device.

[0140] Sub-step 317-a determines a crop size and a crop location for the region of interest diagram based on at least one keypoint of the target to be identified in the bounding box diagram.

[0141] Optionally, the crop size and crop position of the region of interest map are determined based on at least one keypoint based on a reference coordinate system constructed by at least one keypoint and according to a preset distance criterion. The preset distance criterion is associated with the target to be identified and may be set and adjusted based on experience. For example, as shown in FIG. 10 , the x-axis 1001 of the reference coordinate system is constructed by a connecting line between keypoint 2 and keypoint 4, and the y-axis 1002 of the reference coordinate system is constructed by a perpendicular line to the x-axis 1001. The crop size is set to twice the distance between keypoint 2 and keypoint 4 (the region of interest map is rectangular, and the crop size is the size of the side of the region of interest map), and the origin of the reference coordinate system is set to the crop position.

[0142] In sub-step 317-b, the crop position of the region of interest map is adjusted to obtain an adjusted crop position of the region of interest map, and the region of interest map based on the crop size and the adjusted crop position satisfies a first condition in terms of overlap with the target to be identified in the bounding box map.

[0143] After determining the crop size of the region of interest map, the crop position of the region of interest map is adjusted so that the region of interest map covers as many of the identifying features of the target to be identified as possible. For example, as shown in FIG. 10 , the region of interest corresponding to the palm of the hand is adjusted by rotation, translation, etc. to obtain a region of interest 1003 based on the adjusted crop position. The region of interest 1003 based on the adjusted crop position has an overlap with the target to be identified that is greater than an overlap threshold. The overlap threshold may be set and adjusted based on experience.

[0144] In substep 317-c, an initial region of interest diagram is cropped from the bounding box diagram based on the crop size and adjusted crop position.

[0145] As shown in FIG. 10, when image content corresponding to a region of interest 1003 is extracted from the bounding box diagram, an initial region of interest diagram such as initial region of interest diagram 1004 in FIG. 11 is obtained.

[0146] In substep 317-d, the initial region of interest map is scaled to obtain a region of interest map.

[0147] Optionally, the size of the initial region of interest map is scaled down to the required size for input to the quality assessment network to obtain the region of interest map.

[0148] 12, a region of interest image 1201 (e.g., the corresponding number of dimensions is 224*224*3) is input to a quality assessment network 1202. The quality assessment network 1202 evaluates the quality of the region of interest image 1201, thereby obtaining a quality score of the image corresponding to the region of interest map 1201. For example, by sequentially processing the region of interest image 1201 through a convolutional layer, a residual layer (e.g., ResNet-50), and a fully connected layer (2048-dimensional fully connected layer+32-dimensional fully connected layer) in the quality assessment network 1202, a quality score of the image corresponding to the region of interest map 1201 (e.g., the corresponding number of dimensions is 1) can be obtained.

[0149] Optionally, different types of images may be evaluated using a quality assessment network trained on different types of images to improve the accuracy of obtaining quality scores. Alternatively, the quality assessment network may include classifiers corresponding to different types of images to obtain quality scores for different types of images. The multiple classifiers may share a single feature extraction network to reduce the complexity of the quality assessment network.

[0150] In some embodiments, the method for determining the identification result further includes the following steps before obtaining the identification result of the candidate user account based on the N types of weighting coefficients and the N types of image match degrees. The following steps are performed by a computer device. As shown in FIG. 13 , the method includes the following steps before step 330:

[0151] In step 327, feature identification is performed on the i-th type image to identify the i-th type target feature, which is used to represent the variance in feature space of the i-th type image of the target to be identified.

[0152] Optionally, the computing device identifies the image with the feature extraction network to obtain a target feature representation of a target to be identified in the image, the target feature representation being used to represent the target to be identified.

[0153] The target features may be displayed in the form of a vector or a rectangle. The target features may be the "information for expressing an image of the i-th type of target to be identified" mentioned in the above embodiment.

[0154] In step 328, a matching operation is performed on the i-th type target feature and the i-th type candidate feature to obtain the i-th type image match degree, which is used to represent the variance in the feature space of the i-th type images of the candidate user account.

[0155] Optionally, the i-th type of candidate feature is determined in advance. For any one candidate user account, the user of the candidate user account may use a user terminal to provide N types of images obtained by collecting data on the user target using N collection methods to a computer device in advance. The computer device performs feature identification on each of the N types of images uploaded by the candidate user account, and identifies and stores the N types of user features of the candidate user account. Optionally, the candidate features and the target features are acquired in the same manner. That is, the candidate features and the target features exist in the same feature space.

[0156] For example, the computer device is associated with a database, and the N types of user characteristics are stored in the database. The database may be located in the terminal device or in a server, although the present embodiment is not limited thereto.

[0157] Optionally, the step of performing a matching operation between the i-th type target feature and the i-th type candidate feature is realized by calculating a similarity between the i-th type target feature and the i-th type candidate feature. That is, the image match degree refers to the similarity between the i-th type target feature and the i-th type candidate feature. The computer device determines the i-th type image match degree of the candidate user account by representing the similarity between the i-th type target feature and the i-th type candidate feature by the similarity degree. The similarity calculation method includes, but is not limited to, at least one of cosine similarity, Euclidean distance, and Manhattan distance.

[0158] In some embodiments, obtaining an identification result of a candidate user account based on the N types of weighting coefficients and the N types of image match degrees in step 330 may be achieved by computer identification calculating a weighted sum of the N types of image match degrees based on the N types of weighting coefficients to obtain an identification result of a candidate user account.

[0159] Optionally, for a candidate user account, after determining the weight coefficients of the N types of the candidate user account and the image match degrees of the N types of the candidate user account, for an i-th type among the N types, the computer device processes the image match degree of the i-th type using the weight coefficient of the i-th type as the weight of the image match degree of the i-th type to obtain a weighted image match degree of the i-th type, and the computer device sums up the weighted image match degrees respectively corresponding to the N types to obtain an identification result of the candidate user account.

[0160] For example, if the N types include a first type and a second type, and the first type of image is a full-color image and the second type of image is an infrared image, the identification result of a candidate user account may be expressed as follows:

[0161]

number

[0162] where cos(θ) represents the identification result of the candidate user account, α∈[0,1], α represents the first type weight coefficient, and cos(θ r ) represents the first type of image match degree, (1-α) represents the second type of weighting coefficient, and cos(θ i ) represents the second type of image match.

[0163] The imaging quality of different types of images is used to determine different types of weighting coefficients, and the weighting coefficients are used to weight the image match degree of the corresponding type between the target to be identified and the candidate user account, thereby increasing the involvement of images with good imaging quality in the process of confirming the identification result, providing more useful information to the process of confirming the identification result, and contributing to improving the accuracy of the confirmed identification.

[0164] Optionally, after obtaining the quality scores of each of the N types of images, the computer device narrows down the N types of images based on the quality scores of each of the N types of images, and selects M types of images from the N types of images, where M is a positive integer smaller than N and greater than 1. For example, the computer device selects the top M types of images with the highest quality scores from the N types of images, and includes them in the process of calculating the identification results for each of the multiple candidate user accounts. This method reduces the workload of the process of determining the identification results, and contributes to improving the efficiency of obtaining the identification results.

[0165] In one example, if there is a jth type image among the N types of images whose quality score is equal to or greater than the upper score limit, the computer device obtains an identification result for each of the multiple candidate user accounts based on the jth type image of the target to be identified. If there is no jth type image among the N types of images, the method provided in the embodiment of the present application is used to obtain the identification result. The upper score limit is a preset positive number.

[0166] For example, assume that the i-th type of image contains only one image, and a full-color image and an infrared image are collected. If the quality score of the full-color image is greater than the upper score limit and the quality score of the infrared image is less than the upper score limit, the computer device filters out the infrared image and determines the identification results for each of the multiple candidate user accounts directly based on the full-color image. If the quality scores of the full-color image and the infrared image are both less than the upper score limit, the computer device performs steps 320-330 using the method provided in the above embodiment to obtain the identification results.

[0167] In another example, if there is a pth type of image among the N types of images whose quality score is greater than the lower score limit, the computer device obtains a classification result using the method provided in the above embodiment. If the quality scores of each of the N types of images are equal to or less than the lower score limit, the computer device determines that the currently acquired N types of images are invalid. Optionally, the computer device may prompt the user via the image collection device to collect new N types of images of the target to be identified. For example, if full-color images and infrared images are collected and the quality scores of the full-color images and the infrared images are both less than the lower score limit, the computer device determines that the current image collection is invalid. If at least one of the quality scores of the full-color images or the infrared images is greater than the lower score limit, the computer device performs steps 320 to 330 using the method provided in the above embodiment to obtain a classification result.

[0168] In another example, if the quality scores of all N types of images are smaller than the upper score limit and there is a pth type of image whose quality score is greater than the lower score limit, the computer device performs steps 320 to 330 using the method provided in the above embodiment to obtain a classification result. If the quality scores of all N types of images are equal to or less than the lower score limit, the computer device determines that the current image collection is invalid and must prompt the user to collect images again. If there is a jth type of image among the N types of images whose quality score is greater than the upper score limit, the computer device determines the classification result based on the jth type of image.

[0169] For example, if a full-color image, an infrared image, and a depth image are collected, and the quality scores corresponding to the full-color image and the infrared image are all smaller than the upper score limit and larger than the lower score limit, and the quality score corresponding to the depth image is smaller than the lower score limit, the method provided in the above embodiment is used to calculate the weighting coefficients and classification results corresponding to the full-color image, the infrared image, and the depth image, respectively. If the quality scores corresponding to the full-color image, the infrared image, and the depth image are all smaller than the lower score limit, the current image collection is determined to be invalid. If the quality score corresponding to the full-color image is larger than the upper score limit and the quality scores corresponding to the full-color image and the depth image are smaller than the upper score limit, the classification result may be obtained directly based on the full-color image.

[0170] Optionally, if the quality scores of N types of images are all smaller than the upper score limit, and there are pth type images and qth type images whose quality scores are larger than the lower score limit, and there are at least two types of images whose quality scores are smaller than the upper score limit and larger than the lower score limit, the images whose quality scores are smaller than the lower score limit can be filtered, and a classification result can be obtained using the at least two types of images.

[0171] For example, if the quality score corresponding to the full-color image and the quality score corresponding to the infrared image are both smaller than the upper score limit and larger than the lower score limit, and the quality score corresponding to the depth image is smaller than the lower score limit, the depth image can be filtered out and the weighting coefficients and classification results corresponding to the full-color image and the infrared image, respectively, can be calculated using the methods provided in the above examples.

[0172] The lower limit score value is used to indicate whether the imaging quality of the image is poor, and the upper limit score value is used to indicate whether the imaging quality of the image is good. The lower limit score value and the upper limit score value can be set and adjusted based on experience, and the embodiment of the present application is not limited thereto.

[0173] For example, the identification results of each of multiple candidate user accounts are sorted in descending order of the identification result to obtain an identification result series, and the candidate user account corresponding to the first identification result in the identification result series is determined to be the user account that matches the target to be identified.

[0174] In some embodiments, as shown in FIG. 14 , the technical means provided in the embodiments of the present application are configured in a palm scanning device 1401. When a user scans their palm using the palm scanning device 1401, the palm scanning device 1401 periodically collects full-color and infrared images of the palm and selects the best full-color and infrared images from the collected images. The palm scanning device 1401 then identifies the living body performing the palm scan based on the best infrared image, obtains a quality score for the full-color image and a quality score for the infrared image, and determines a weighting factor for the full-color image and a weighting factor for the infrared image. The palm scanning device 1400 then obtains the similarity between the full-color image and each preliminary feature representation in the database, and the similarity between the infrared image and each preliminary feature representation in the database, respectively. Then, based on the weighting factor for the full-color image and the weighting factor for the infrared image, it calculates a weighted sum of the similarity corresponding to the full-color image and the similarity corresponding to the infrared image, thereby obtaining a classification result between the palm and each preliminary feature representation. Finally, it determines a user account matching the palm based on each classification result. If it is determined that a user account exists in the database that matches the target to be identified, the palm scan is deemed successful; otherwise, the palm scan is deemed unsuccessful.

[0175] Next, the training process of the quality assessment model will be described. Optionally, the target detection network, the keypoint detection network, and the quality assessment network in the quality assessment model can be trained separately, or the quality assessment model can be trained as a whole. However, the present embodiment is not limited thereto.

[0176] In some embodiments, the training process of the target detection network may include the following:

[0177] A large number of sample images annotated with label data, for example, images of different types annotated with palm label data, are obtained. A target detection network is used to obtain predicted bounding boxes corresponding to each sample image, and a training loss for the target detection network is obtained based on the predicted bounding boxes corresponding to each sample image and the label data. Finally, the target detection network is repeatedly trained using the training loss for the target detection network, and a fully trained target detection network is obtained.

[0178] In some embodiments, the training process of the keypoint detection network may include the following:

[0179] A large number of sample images with labeled data are obtained, for example, different types of bounding box diagrams with palm keypoint labeled data. A keypoint detection network is used to obtain predicted keypoints corresponding to each sample image. The training loss of the keypoint detection network is then obtained based on the predicted keypoints and label data corresponding to each sample image. Finally, the keypoint detection network is repeatedly trained based on the training loss, resulting in a fully trained keypoint detection network.

[0180] In some embodiments, the quality assessment network training process may include the following:

[0181] 1. Acquire multiple sample image sets, each of which contains multiple sample images corresponding to the same sample target.

[0182] For example, for one user, multiple images of different types of interest for the palm of the hand are acquired, and for M users, M sample image sets are acquired.

[0183] 2. For each sample image set among the plurality of sample image sets, obtain a feature representation of the sample target in each sample image among the sample image sets.

[0184] Optionally, a discrimination score evaluation model performs feature extraction on each sample image to obtain a feature representation of the sample target in each sample image. The discrimination score evaluation model is used to obtain the discrimination score. Optionally, the discrimination score evaluation model may be based on FaceQnet or a proprietary feature algorithm.

[0185] For example, as shown in FIG. 15, a sample image set 1501 includes N sample images, and when feature extraction is performed on each sample image using a discrimination point evaluation model 1502, feature expressions corresponding to each of the N sample images are obtained.

[0186] 3. Obtain the target sample image with the best imaging quality among the sample images.

[0187] Optionally, the target sample images may be obtained by human labeling or by classification using a feature algorithm, although the present embodiment is not limited thereto.

[0188] 4. Calculate the similarity between the feature representations corresponding to other sample images in the sample image set and the feature representation corresponding to the target sample image, and use this as the discrimination score corresponding to the other sample images.

[0189] The other sample images in the sample image set refer to sample images in the sample image set excluding the target sample image. Optionally, the similarity calculation method may be, for example, cosine similarity, Euclidean distance, Manhattan distance, etc. The discrimination score is used to represent the discrimination accuracy of the discrimination score evaluation model for the sample target and the similarity between the sample target in the other sample images and the sample target in the target sample image.

[0190] For example, as shown in FIG. 15, after obtaining the feature representation corresponding to the target sample image in the sample image set 1501 and the feature representation corresponding to the other sample images using the discrimination score evaluation model 1502, the similarity between the feature representation corresponding to the other sample images and the feature representation corresponding to the target sample image is calculated using Euclidean distance, and discrimination scores corresponding to the other sample images are obtained.

[0191] When the range of the discrimination points is 0 to 1, the discrimination point number corresponding to the target sample image is 1. Then, the N sample images and the discrimination points corresponding to the N sample images are stored in association with each other in the database.

[0192] 5. Using the quality assessment network, a predicted quality score corresponding to each sample image in the set of multiple sample images is obtained.

[0193] The predicted quality score refers to the quality score corresponding to the sample image. When the quality assessment network in Figure 12 is used to assess the quality of each sample image, a predicted quality score corresponding to each sample image is obtained.

[0194] 6. Train a quality assessment network based on the predicted quality scores corresponding to each sample image in the set of multiple sample images and the discrimination scores corresponding to each sample image in the set of multiple sample images, to obtain a trained quality assessment network.

[0195] Optionally, the quality assessment network can be trained under supervision using the classification scores corresponding to each sample image as label data corresponding to each sample image, to obtain a fully trained quality assessment network.

[0196] For example, first, a training loss of the quality assessment network is determined based on the predicted quality scores corresponding to each sample image in the set of sample images and the classification scores corresponding to each sample image in the set of sample images. The training loss may be expressed as follows:

[0197]

number

[0198] where Y is the discrimination score, f(x) is the predicted quality score corresponding to sample image x, and N is the total number of sample images.

[0199] The parameters of the quality assessment network are iteratively adjusted with the goal of minimizing the training loss of the quality assessment network, resulting in a fully trained quality assessment network.

[0200] According to the similarity between the feature representation of a sample target in a different sample image and the feature representation of the sample target in a corresponding target sample image with high imaging quality, the larger the similarity, the higher the imaging quality of the sample image, and the smaller the similarity, the lower the imaging quality of the sample image, which means that there is a correlation between the imaging quality of the sample image and the above-mentioned discrimination score. By training the quality assessment network with a large number of sample images with discrimination scores, the quality score distribution of all sample images in the quality assessment network will eventually be similar to the discrimination score distribution. This can be considered an indication that the quality assessment network's assessment of imaging quality is accurate.

[0201] Next, we will describe apparatus embodiments that can be used to implement the method embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, please refer to the method embodiments of the present application.

[0202] FIG. 16 shows a block diagram of an apparatus for determining a recognition result provided in one embodiment of the present application. The apparatus has functions for implementing the above-described method example. The functions may be implemented by hardware or by executing corresponding software on the hardware. The apparatus may be the computer device described above or may be installed within the computer device. As shown in FIG. 16, the apparatus 1600 includes a multiple-type image acquisition module 1601, a weighting coefficient acquisition module 1602, a recognition result acquisition module 1603, and a user account matching module 1604.

[0203] The multiple type image acquisition module 1601 is used to acquire N types of images acquired by performing acquisition on a target to be identified, where different types of images are acquired using different acquisition methods, and N is an integer greater than 1.

[0204] The weighting coefficient acquisition module 1602 is used to determine a weighting coefficient for the i-th type among the N types based on the imaging quality of the image of the i-th type, where i is a positive integer less than or equal to N.

[0205] The identification result acquisition module 1603 is used to obtain an identification result for each of a plurality of candidate user accounts based on the N types of weighting coefficients and the N types of image match degrees, where the i type of image match degree is used to represent the feature match degree between the target to be identified and the candidate user account with respect to the i type of image.

[0206] The user account matching module 1604 is used to determine a user account that matches the target to be identified from among the plurality of candidate user accounts based on the identification result of each of the plurality of candidate user accounts.

[0207] In some embodiments, as shown in FIG. 17 , the weighting factor obtaining module 1602 includes a quality score determination sub-module 1602a, a weighting parameter obtaining sub-module 1602b, and a weighting factor obtaining sub-module 1602c. The quality score determination sub-module 1602a is used to determine a quality score of the i-type image, and the quality score of the i-type image is used to represent the imaging quality of the i-type image. The weighting parameter obtaining sub-module 1602b is used to determine a weighting characteristic parameter of the i-type image based on a numerical relationship between the quality score of the i-type image and a quality score threshold. The weighting characteristic parameter of the i-type image is used to control the influence of the i-type image on the classification result. The weighting factor obtaining sub-module 1602c is used to perform a weighting factor calculation based on the weighting characteristic parameter of the i-type image to obtain the i-type weighting factor.

[0208] In some embodiments, the weight parameter acquisition sub-module 1602b is used to determine the smaller value of the quality score of the i type image and the quality score threshold as the weight characteristic parameter of the i type, or to determine the larger value of the quality score of the i type image and the quality score threshold as the weight characteristic parameter of the i type, or to determine the average value of the quality score of the i type image and the quality score threshold as the weight characteristic parameter of the i type.

[0209] In some embodiments, the weight coefficient acquisition sub-module 1602c is used to sum up the weight characteristic parameters of each of the N types to obtain a sum result, and then perform a ratio calculation between the weight characteristic parameter of the i type and the sum result to obtain the weight coefficient of the i type.

[0210] In some embodiments, the device 1600 further includes a match degree determination module 1605. The match degree determination module 1605 is used to perform feature identification on the i-type image to identify the i-type target features, and to perform a matching operation between the i-type target features and the i-type candidate features to obtain the i-type image match degree. The i-type target features are used to represent the variance in feature space of the i-type images of the target to be identified, and the i-type candidate features are used to represent the variance in feature space of the i-type images of the candidate user account.

[0211] In some embodiments, the identification result acquisition module 1603 is used to obtain an identification result of the candidate user account by calculating a weighted sum of the image match degrees of the N types based on the weight coefficients of the N types.

[0212] In some embodiments, the imaging quality is determined by a quality assessment model. The quality assessment model includes a target detection network, a keypoint detection network, and a quality assessment network. The apparatus 1600 further includes a bounding box detection module 1606, a keypoint detection module 1607, an interest acquisition module 1608, and a quality score determination module 1609. The bounding box detection module 1606 is used to perform target detection on the i-type image using the target detection network to determine a bounding box diagram of an image of the target to be identified in the i-type image. The bounding box diagram indicates an imaging area of ​​the target to be identified in the i-type image. The keypoint detection module 1607 is used to perform keypoint detection on the bounding box diagram using the keypoint detection network to determine at least one keypoint of the target to be identified in the bounding box diagram. The interest acquisition module 1608 is used to crop the bounding box diagram based on the at least one keypoint to obtain a region of interest diagram of the target to be identified. The region of interest map includes discriminative features for identifying the target to be identified. A quality score determination module 1609 is used to perform quality assessment on the region of interest map using the quality assessment network to determine a quality score of the i-type image. The quality score of the i-type image is used to represent the imaging quality of the i-type image.

[0213] In some embodiments, the interest acquisition module 1608 is used to determine a crop size and a crop position of the region of interest diagram based on at least one key point of the target to be identified in the bounding box diagram, adjust the position relative to the crop position of the region of interest diagram to obtain an adjusted crop position of the region of interest diagram, crop an initial region of interest diagram from the bounding box diagram based on the crop size and the adjusted crop position, and perform scaling on the initial region of interest diagram to obtain the region of interest diagram, wherein the degree of overlap between the region of interest diagram based on the crop size and the adjusted crop position and the target to be identified in the bounding box diagram satisfies a first condition.

[0214] In some embodiments, the training process of the quality assessment network includes the following steps: acquiring a plurality of sample image sets, each of which includes a plurality of sample images corresponding to the same sample target; acquiring, for each of the plurality of sample image sets, a feature representation of the sample target for each sample image in the sample image set; acquiring a target sample image with the best imaging quality among the sample images; calculating the similarity between the feature representations corresponding to the other sample images in the sample image set and the feature representation corresponding to the target sample image, and using this as a discrimination score corresponding to the other sample image; acquiring a predicted quality score corresponding to each of the plurality of sample image sets based on the quality assessment network; training the quality assessment network based on the predicted quality scores corresponding to each of the sample images in the plurality of sample image sets and the discrimination scores corresponding to each of the sample images in the plurality of sample image sets to obtain the trained quality assessment network.

[0215] In some embodiments, the N types of images include a full color image and an infrared image.

[0216] As described above, the technical means provided in the present embodiment obtains multiple different types of images corresponding to the target to be identified, determines a weighting factor for each type of image based on the imaging quality corresponding to each of the multiple different types of images, and then combines the images based on the weighting factor for each type of image to identify a user account that matches the target to be identified from the database. In this way, the present embodiment rationally and accurately assigns a weighting factor for each type of image based on the imaging quality of the image, thereby improving the accuracy of the identification result and ultimately the accuracy of target identification. Particularly in situations with complex lighting, rationally and accurately assigning a weighting factor for each type of image can increase the impact of images with good imaging quality on the identification result and decrease the impact of images with poor imaging quality on the identification result, thereby improving the accuracy of target identification.

[0217]

[0023] To further explain, although the above embodiments only describe examples in which the functions of the device are realized by dividing the functional modules, in actual operation, the functions may be divided and performed by different functional modules as needed. That is, the internal structure of the device may be divided into different functional modules to perform all or part of the above-described functions. Furthermore, since the device embodiments provided in the above embodiments are based on the same concept as the method embodiments, please refer to the method embodiments for specific implementation processes, and they will not be described again here.

[0218] 18 shows a structural block diagram of a computer device provided in one embodiment of the present application, which can be used to implement the method for determining the identification result provided in the above embodiment, and specifically includes the following contents:

[0219] The computer device 1800 includes a central processing unit (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or a field programmable gate array (FPGA)) 1801, a system memory 1804 including random-access memory (RAM) 1802 and read-only memory (ROM) 1803, and a system bus 1805 connecting the system memory 1804 and the central processing unit 1801. The computer device 1800 further includes an input / output system (I / O system) 1806 that supports information transmission between devices within the server, and a mass storage device 1807 for storing an operating system 1813, applications 1814, and other program modules 1815.

[0220] The basic input / output system 1806 includes a display 1808 for displaying information and input devices 1809, such as a mouse or keyboard, for inputting information by a user. The display 1808 and input devices 1809 are connected to the central processing unit 1801 through an input / output controller 1810, which is connected to the system bus 1805. The basic input / output system 1806 may also include an input / output controller 1810 for receiving and processing input from a number of other devices, such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller 1810 may provide output to a display screen, a printer, or other type of output device.

[0221] The mass storage device 1807 is connected to the central processing unit 1801 via a mass storage controller (not shown) connected to the system bus 1805. The mass storage device 1807 and its associated computer-readable media provide non-volatile storage for the computing device 1800. That is, the mass storage device 1807 may also include another computer-readable medium (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0222] Without loss of generality, such computer-readable media may include computer storage media and communication media. Computer storage media includes volatile and nonvolatile, mobile and non-mobile media embodied in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory, or other solid-state storage technology, CD-ROM, Digital Video Disc (DVD) or other optical memory, cassette tape, magnetic tape, magnetic disk memory, or other magnetic storage devices. Of course, those skilled in the art will recognize that such computer storage media are not limited to the above types. The system memory 1804 and mass storage device 1807 may be collectively referred to as memory.

[0223] According to an embodiment of the present invention, the computing device 1800 may operate by connecting to a remote computer over a network, such as the Internet, for example. That is, the computing device 1800 may be connected to a network 1812 via a network interface unit 1811 connected to the system bus 1805. Alternatively, the computing device 1800 may use the network interface unit 1811 to connect to other types of networks or remote computer systems (not shown).

[0224] The memory further includes a computer program that is stored in the memory and configured to be executed by one or more processors to implement the method for determining the identification result.

[0225] In some embodiments, a computer-readable storage medium is further provided, which stores a computer program, which, when executed by a processor, implements the method for determining an identification result.

[0226] Optionally, the computer-readable storage medium includes ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives), or optical disk, etc. Among them, the Random-Access Memory includes ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0227] In some exemplary embodiments, a computer program product is further provided, the computer program product including a computer program stored in a computer-readable storage medium, the computer program being read by a processor of a computing device from the computer-readable storage medium and executed by the processor to cause the computing device to perform the method for determining an identification result.

[0228] For additional information, in the present embodiment, before and during the collection of user-related data, a prompt interface or pop-up window is displayed or voice prompt information is output to inform the user that user-related data is currently being collected. Only after the user confirms the prompt interface or pop-up window does the relevant step of acquiring user-related data begin. Otherwise (i.e., if the user does not confirm the prompt interface or pop-up window), the relevant step of acquiring user-related data is terminated and user-related data is not acquired. In other words, all user data collected in this application is handled in strict accordance with the provisions of relevant national laws and regulations, and the informed consent or individual agreement of the personal information subject is obtained. All collection is conducted under conditions of user consent and authentication. Subsequent data use and processing activities are conducted within the scope of laws and regulations and the authentication of the personal information subject. The collection, use, and processing of relevant user data must also comply with the relevant laws, regulations, and standards of relevant countries and regions. For example, the N different types of images, targets to be identified, and sample image sets in this application are all collected under fully authenticated conditions.

[0229] Note that "multiple" as referred to in this specification means two or more than two. "And / or" indicates a relationship between related objects, and indicates that three types of relationships exist. For example, "A and / or B" indicates three situations: a situation in which only A exists, a situation in which A and B exist simultaneously, and a situation in which only B exists. The symbol " / " generally indicates that the related objects before and after it are in an "or" relationship. Note that the step numbers described in this specification are merely illustrative of possible execution orders between steps. In some other embodiments, the steps may be executed out of the order of the numbers, for example, by executing two differently numbered steps simultaneously or by executing two differently numbered steps in the reverse order of the illustration. The embodiments of the present application are not limited thereto.

Claims

1. 1. A method for determining an identification result executed by a computer device, comprising: acquiring N types of images (N is an integer greater than 1) acquired from a target to be identified; determining a weighting coefficient for an i-th type (i is a positive integer equal to or less than N) among the N types based on an imaging quality of the i-th type image; For each of the plurality of candidate user accounts, obtaining an identification result of the candidate user account based on the N types of weight coefficients and the N types of image match degrees; determining a user account that matches the target to be identified from among the plurality of candidate user accounts based on the identification result of each of the plurality of candidate user accounts; The different types of images were obtained using different acquisition methods, The image match degree of the i type is used to represent a feature match degree between the target to be identified and the candidate user account regarding the image of the i type. How to confirm the identification results.

2. The step of determining a weighting factor of the i-type based on the imaging quality of the i-type image comprises: determining a quality score for the i-type image, the quality score being used to represent the imaging quality of the i-type image; determining a weight characteristic parameter of the i-type image based on a numerical relationship between the quality score of the i-type image and a quality score threshold, for controlling the influence of the i-type image on the classification result; calculating a weight coefficient based on the weight characteristic parameter of the i-th type to obtain the weight coefficient of the i-th type; The method for determining an identification result according to claim 1 .

3. determining a weight characteristic parameter of the i-th type based on a numerical relationship between the quality score of the i-th type image and a quality score threshold, determining a smaller value of the quality score of the image of the i-type and the quality score threshold as a weight characteristic parameter of the i-type; or determining a larger value of the quality score of the image of the i-type and the quality score threshold as a weight characteristic parameter of the i-type; or determining a weight characteristic parameter of the i-type to be an average value of the quality score of the i-type image and the quality score threshold; The method for determining an identification result according to claim 2.

4. The step of calculating a weight coefficient based on the weight characteristic parameter of the i-th type to obtain the weight coefficient of the i-th type includes: summing the weight characteristic parameters of each of the N types to obtain a sum; performing a ratio calculation between the weight characteristic parameter of the i-th type and the sum result to obtain a weight coefficient of the i-th type; 4. A method for determining an identification result according to claim 2 or 3.

5. before the step of obtaining an identification result of the candidate user account based on the N types of weight coefficients and the N types of image match degrees, performing feature identification on the image of the i-th type to identify a target feature of the i-th type; performing a matching operation on the target feature of the i type and the candidate feature of the i type to obtain an image match degree of the i type; The i-type target feature is used to represent the variance in a feature space of the i-type image of the target to be identified; the i-type candidate feature is used to represent the variance in the feature space of the i-type images of the candidate user account; A method for determining an identification result according to any one of claims 1 to 4.

6. The step of obtaining the candidate user account identification result based on the N types of weight coefficients and the N types of image match degrees includes: calculating a weighted sum of the image match degrees of the N types based on the weight coefficients of the N types to obtain an identification result of the candidate user account; A method for determining an identification result according to any one of claims 1 to 5.

7. The imaging quality is determined by a quality assessment model, and the quality assessment model includes a target detection network, a keypoint detection network, and a quality assessment network, and the method for determining the identification result further includes: performing target detection on the image of the i-type using the target detection network, and determining a bounding box diagram of the image of the target to be identified in the image of the i-type; performing keypoint detection on the bounding box diagram using the keypoint detection network to determine at least one keypoint of the target to be identified in the bounding box diagram; cropping the bounding box diagram based on the at least one key point to obtain a region of interest diagram of the target to be identified; performing a quality assessment on the region of interest map by the quality assessment network to determine a quality score for the i-type image to represent an imaging quality of the i-type image; The bounding box diagram is an imaging area of ​​the target to be identified in the image of the i type; the region of interest map includes a distinguishing feature for identifying the target to be distinguished; A method for determining an identification result according to any one of claims 1 to 6.

8. The step of cropping the bounding box diagram based on the at least one key point to obtain a region of interest diagram of the target to be identified includes: determining a crop size and a crop position of the region of interest map based on the at least one key point; performing a position adjustment on the crop position of the region of interest map to obtain an adjusted crop position of the region of interest map; cropping an initial region of interest diagram from the bounding box diagram based on the crop size and the adjusted crop position; scaling the initial region of interest map to obtain the region of interest map; an overlapping degree of the region of interest diagram based on the crop size and the adjusted crop position with the target to be identified in the bounding box diagram satisfies a first condition; The method for determining an identification result according to claim 7.

9. The training process of the quality assessment network includes: acquiring a plurality of sample image sets each including a plurality of sample images corresponding to the same sample target; For each of the plurality of sample image sets, obtaining a feature representation of the sample target in each sample image of the sample image set; acquiring a target sample image having the best imaging quality among the sample images; a step of calculating similarities between feature representations corresponding to other sample images in the sample image set and feature representations corresponding to the target sample image, and setting the similarities as discrimination scores corresponding to the other sample images; obtaining, by the quality assessment network, a predicted quality score corresponding to each sample image in the set of sample images; training the quality assessment network based on the predicted quality scores corresponding to each sample image in the set of sample images and the classification scores corresponding to each sample image in the set of sample images to obtain a trained quality assessment network; 9. A method for determining an identification result according to claim 7 or 8.

10. The N types of images include full color images and infrared images. A method for determining an identification result according to any one of claims 1 to 9.

11. a multiple type image acquisition module for acquiring N types of images (N being an integer greater than 1) acquired from a target to be identified; a weighting coefficient obtaining module for determining a weighting coefficient of an i-th type (i is a positive integer equal to or less than N) among the N types based on an imaging quality of the i-th type; an identification result acquisition module for acquiring an identification result of each candidate user account among a plurality of candidate user accounts based on the N types of weight coefficients and the N types of image match degrees; a user account matching module for determining a user account that matches the target to be identified from among the plurality of candidate user accounts based on the identification results of each of the plurality of candidate user accounts; The different types of images were obtained using different acquisition methods, The image match degree of the i type represents a feature match degree between the target to be identified and the candidate user account regarding the image of the i type. A device for determining identification results.

12. A computer device including a processor and a memory, wherein a computer program is stored in the memory, and the method for determining an identification result according to any one of claims 1 to 10 is realized by loading and executing the computer program by the processor. Computer equipment.

13. A computer-readable storage medium storing a computer program, the computer program being loaded and executed by a processor to realize the method for determining an identification result according to any one of claims 1 to 10. A computer-readable storage medium.

14. A computer program product including a computer program, the computer program being stored in a computer-readable storage medium, and the method for determining an identification result according to any one of claims 1 to 10 being realized when a processor reads and executes the computer program from the computer-readable storage medium. Computer program products.

Images