Unlocking method, electronic device, storage medium and computer program product

Abstract

Embodiments of the present application provide a kind of unlocking method, electronic equipment, storage medium and computer program product.The method comprises: obtaining the current unlocking image corresponding to current unlocking operation;At least one previous unlocking image corresponding to at least one previous unlocking operation before current unlocking operation is obtained;From at least one previous unlocking image, find the previous unlocking image between current unlocking image, and the first similarity between them is greater than the first similarity threshold value;Determine whether to unlock based on the previous unlocking result corresponding to the plurality of previous unlocking images respectively found, or based on the current comparison result and the previous unlocking result corresponding to one or more previous unlocking images respectively found, wherein, current comparison result is the comparison result obtained by comparing current unlocking image with target base library in base library image. Help to improve the accuracy of biometric identification, in turn help to improve the success rate of unlocking.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically to an unlocking method, electronic device, storage medium, and computer program product. Background Technology

[0002] Currently, many electronic devices employ various unlocking technologies for security reasons, requiring users to verify their identity before unlocking the device. Existing unlocking technologies include facial recognition, fingerprint unlocking, and other biometric-based technologies.

[0003] Existing unlocking technologies have some limitations in use. For example, the limited area or performance of biometric data collection, such as facial features or fingerprints, results in low accuracy in recognizing these biometric features, thus reducing the success rate of unlocking. Summary of the Invention

[0004] This application is made in view of the above-mentioned problems. This application provides an unlocking method, an electronic device, a storage medium, and a computer program product.

[0005] According to one aspect of this application, an unlocking method is provided, comprising: acquiring a current unlocking image corresponding to a current unlocking operation; acquiring at least one previous unlocking image corresponding to at least one previous unlocking operation prior to the current unlocking operation; searching for a previous unlocking image from the at least one previous unlocking image whose first similarity to the current unlocking image is greater than a first similarity threshold; determining whether to unlock based on the previous unlocking results corresponding to each of the found previous unlocking images, or based on the current comparison result and the previous unlocking results corresponding to each of the found one or more previous unlocking images, wherein the current comparison result is a comparison result obtained by comparing the current unlocking image with images in a target database.

[0006] For example, the current comparison result includes a second similarity between the currently unlocked image and each of one or more images in the target database. Based on the current comparison result and the previous unlock results corresponding to one or more previously unlocked images, determining whether to unlock includes: determining a current unlock threshold based on the previous unlock results corresponding to one or more previously unlocked images; if there is a second similarity greater than the current unlock threshold in the current comparison result, then unlocking is determined to be successful; if there is no second similarity greater than the current unlock threshold in the current comparison result, then unlocking is determined to be unsuccessful.

[0007] For example, determining the current unlock threshold based on the previous unlock results corresponding to one or more previously unlocked images found includes: determining auxiliary unlock information based on the previous unlock results corresponding to one or more previously unlocked images found; and determining the current unlock threshold based on the auxiliary unlock information.

[0008] For example, determining the current unlock threshold based on the auxiliary unlock information includes: increasing the initial unlock threshold to obtain the current unlock threshold when the auxiliary unlock information indicates that the device should not be unlocked; and / or, decreasing the initial unlock threshold to obtain the current unlock threshold when the auxiliary unlock information indicates that the device should be unlocked.

[0009] For example, determining the auxiliary unlocking information based on the previous unlocking results corresponding to one or more previously unlocked images includes: if a first preset number or a first preset proportion of the previous unlocking results corresponding to one or more previously unlocked images are successfully unlocked, then the auxiliary unlocking information is determined to be unlocked; otherwise, the auxiliary unlocking information is determined to be not unlocked.

[0010] For example, the previous unlocking result includes a second similarity between the corresponding previously unlocked image and the images in the target database. Based on the previous unlocking results corresponding to one or more previously unlocked images, determining the auxiliary unlocking information includes: taking a weighted average of the second similarities corresponding to one or more previously unlocked images to obtain a first average similarity; if the first average similarity is greater than the second similarity threshold, the auxiliary unlocking information is determined to be unlocked, otherwise the auxiliary unlocking information is determined to be not unlocked.

[0011] For example, in the weighted average, for any previously unlocked image among one or more previously unlocked images found, the closer the execution time of the previous unlocking operation corresponding to the previously unlocked image is to the execution time of the current unlocking operation, the greater the weight of the second similarity corresponding to the previously unlocked image.

[0012] For example, the previous unlocking result includes the previous comparison result. The current comparison result and the previous comparison result each include a second similarity between the corresponding unlocked image and each of the one or more images in the target database. Based on the current comparison result and the previous unlocking results corresponding to the one or more previously unlocked images, determining whether to unlock includes: taking a weighted average of the second similarities corresponding to the one or more previously unlocked images and the current unlocked image to obtain a second average similarity; comparing the second average similarity with a current unlocking threshold; if the second average similarity is greater than the current unlocking threshold, then unlocking is determined to be successful; if the second average similarity is not greater than the current unlocking threshold, then unlocking is determined to be unsuccessful.

[0013] For example, in the weighted average, the second similarity corresponding to the current unlocked image has the largest weight, and for any of the previously unlocked images found, the closer the execution time of the previous unlocking operation corresponding to the previous unlocked image is to the execution time of the current unlocking operation, the greater the weight of the second similarity corresponding to the previous unlocked image.

[0014] For example, the current comparison result includes a second similarity between the current unlocked image and each of one or more images in the target database. After finding a previously unlocked image from at least one previously unlocked image whose first similarity to the current unlocked image is greater than a first similarity threshold, the method further includes: if no one or more previously unlocked images are found, determining the current unlock threshold as an initial unlock threshold; if there is a second similarity greater than the current unlock threshold in the current comparison result, determining that the unlocking is successful; if there is no second similarity greater than the current unlock threshold in the current comparison result, determining that the unlocking has failed.

[0015] For example, determining whether to unlock based on the previous unlock results corresponding to each of the found previously unlocked images includes one of the following: if the previous unlock results corresponding to each of the found previously unlocked images are all successful, then unlocking is determined to be successful; otherwise, unlocking is determined to be unsuccessful; if a second preset number or a second preset proportion of the found previously unlocked images have previous unlock results corresponding to successful unlocking, then unlocking is determined to be successful; otherwise, unlocking is determined to be unsuccessful; if a third preset number or a third preset proportion of the found previously unlocked images have previous unlock results corresponding to successful unlocking, then unlocking is determined to be successful; otherwise, unlocking is determined to be unsuccessful; if the previous unlock results corresponding to each of the found previously unlocked images are all successful, and the found previously unlocked images... If the previously unlocked image was acquired within a first preset time period prior to the current time, then unlocking is determined to be successful; otherwise, unlocking is determined to be unsuccessful. If among the multiple previously unlocked images found, there is a fourth preset number or a fourth preset proportion of previously unlocked images whose previous unlocking result is successful, and the fourth preset number or a fourth preset proportion of previously unlocked images were acquired within a second preset time period prior to the current time, then unlocking is determined to be successful; otherwise, unlocking is determined to be unsuccessful. If among the multiple previously unlocked images found, there is a fifth preset number or a fifth preset proportion of consecutive previously unlocked images whose previous unlocking result is successful, and the fifth preset number or a fifth preset proportion of consecutive previously unlocked images were acquired within a third preset time period prior to the current time, then unlocking is determined to be successful; otherwise, unlocking is determined to be unsuccessful.

[0016] For example, after obtaining the current unlock image corresponding to the current unlock operation, the method further includes: selecting representative images from multiple previous image sets or multiple base libraries that correspond one-to-one with multiple different target objects to obtain representative image sets corresponding to each of the multiple different target objects; comparing the current unlock image with each representative image set to determine a third similarity between the current unlock image and each representative image set; selecting the previous image set and base library corresponding to the representative image set with the largest third similarity; determining the image in the selected previous image set as at least one previous unlock image; and determining the selected base library as the target base library.

[0017] For example, at least one previously unlocked image is stored in a storage unit, and the number of previously unlocked images stored in the storage unit is fixed.

[0018] For example, the method further includes: determining the image quality of previously unlocked images stored in the storage unit; deleting the target previously unlocked image and the corresponding previously unlocked result from the storage unit, and storing the current unlocked image and the current unlocked result corresponding to the current unlocked image in the storage unit, wherein the target previously unlocked image is the image with the lowest image quality among the previously unlocked images stored in the storage unit.

[0019] For example, at least one previously unlocked image is a previously unlocked image acquired within a fourth preset time period prior to the current moment.

[0020] According to another aspect of this application, an electronic device is provided, including a processor and a memory, wherein the memory stores computer program instructions, which are executed by the processor to perform the above-described unlocking method.

[0021] According to another aspect of this application, a storage medium is provided on which program instructions are stored, which, when executed, are used to perform the above-described unlocking method.

[0022] According to another aspect of this application, a computer program product is provided, the computer program product comprising a computer program that, when run, performs the above-described unlocking method.

[0023] The unlocking method, electronic device, storage medium, and computer program product according to embodiments of this application can make unlocking decisions based on information from multiple frames of unlocking images rather than information from a single frame, allowing unlocking to rely on more image information. Therefore, this approach helps improve the accuracy of biometric recognition, thereby increasing the success rate of unlocking. Attached Figure Description

[0024] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0025] Figure 1 A schematic block diagram of an example electronic device for implementing the unlocking method and apparatus according to embodiments of this application is shown;

[0026] Figure 2 A schematic flowchart illustrating an unlocking method according to an embodiment of this application is shown;

[0027] Figure 3 A flowchart illustrating an unlocking method according to an embodiment of this application is shown;

[0028] Figure 4 A schematic block diagram of an unlocking device according to an embodiment of this application is shown;

[0029] Figure 5 A schematic block diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0030] In recent years, significant progress has been made in research on technologies based on artificial intelligence, such as computer vision, deep learning, machine learning, image processing, and image recognition. Artificial intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies, and application systems to simulate and extend human intelligence. AI is a comprehensive discipline involving numerous technologies, including chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, and neural networks. Computer vision, as an important branch of AI, specifically enables machines to recognize the world. Computer vision technologies typically include facial recognition, unlocking, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, object detection, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, and robot navigation and localization. With the research and advancement of artificial intelligence technology, this technology has been applied in numerous fields, such as security and prevention, urban management, traffic management, building management, park management, facial recognition access control, facial recognition attendance, logistics management, warehouse management, robotics, intelligent marketing, computational photography, mobile imaging, cloud services, smart homes, wearable devices, autonomous driving, autonomous driving, smart healthcare, facial recognition payment, facial recognition unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile internet, live streaming, beauty filters, cosmetics, medical aesthetics, and intelligent temperature measurement.

[0031] To make the objectives, technical solutions, and advantages of this application more apparent, exemplary embodiments according to this application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein. Based on the embodiments of this application described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of this application.

[0032] As mentioned above, the limited area or performance of biometric sensors such as faces and fingerprints result in low accuracy in recognizing these features, thus reducing the success rate of unlocking. The following explanation uses fingerprint unlocking as an example. Fingerprint unlocking involves the fingerprint recognition process, which determines whether two fingerprint images belong to the same finger; only if they do can the phone unlock. Fingerprint unlocking technology is commonly used in smart terminals such as mobile phones. Many manufacturers now propose placing the fingerprint unlocking module on the side of the phone, which presents a significant challenge to fingerprint unlocking technology. Compared to traditional fingerprint unlocking, the main difficulties of side-mounted fingerprint unlocking are: 1. Smaller image area; 2. Larger aspect ratio, leading to uneven signal distribution.

[0033] To at least partially address the aforementioned technical problems, embodiments of this application provide an unlocking method, an electronic device, a storage medium, and a computer program product. According to the unlocking method of this application, unlocking information from multiple consecutive unlocking frames is integrated, and the current unlocking operation is judged based on the integrated information rather than the information from a single unlocking frame. This allows unlocking to rely on more image information, thereby helping to improve the accuracy of biometric recognition such as fingerprints and faces, and increasing the success rate of unlocking. Experiments show that when this technology is applied to the field of fingerprint unlocking, it can significantly improve the accuracy of fingerprint recognition, especially the accuracy of side fingerprint recognition.

[0034] Although the above description uses fingerprint unlocking, especially side-mounted fingerprint unlocking, as an example to further illustrate the problems with existing unlocking technologies, it is understood that the unlocking technology according to the embodiments of this application can be applied to any field requiring unlocking, including but not limited to various technical fields based on face, fingerprint, iris, or other features for unlocking. When the unlocking method described herein is applied to the field of fingerprint unlocking, it can be applied to contact fingerprint unlocking or contactless (e.g., optical) fingerprint unlocking. When the unlocking method described herein is applied to the field of fingerprint unlocking, it can be applied to frontal fingerprint unlocking or side-mounted fingerprint unlocking.

[0035] First, refer to Figure 1 This describes an example electronic device 100 for implementing the unlocking method and apparatus according to embodiments of this application.

[0036] like Figure 1 As shown, the electronic device 100 includes one or more processors 102 and one or more storage devices 104. Optionally, the electronic device 100 may also include an input device 106, an output device 108, and an image acquisition device 110, these components being interconnected via a bus system 112 and / or other forms of connection mechanisms (not shown). It should be noted that... Figure 1 The components and structure of the electronic device 100 shown are merely exemplary and not limiting; the electronic device may also have other components and structures as needed.

[0037] The processor 102 may be implemented in at least one of the following hardware forms: digital signal processor (DSP), field-programmable gate array (FPGA), programmable logic array (PLA), and microprocessor. The processor 102 may be one or a combination of several of the following: central processing unit (CPU), graphics processing unit (GPU), application-specific integrated circuit (ASIC), or other processing units with data processing capabilities and / or instruction execution capabilities. It may also control other components in the electronic device 100 to perform the desired functions.

[0038] The storage device 104 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 102 may execute the program instructions to implement the client functions (implemented by the processor) in the embodiments of this application described below, and / or other desired functions. Various applications and various data may also be stored in the computer-readable storage medium, such as various data used and / or generated by the applications.

[0039] The input device 106 may be a device used by a user to input commands, and may include one or more of the following: keyboard, mouse, microphone, and touch screen.

[0040] The output device 108 can output various information (e.g., images and / or sound) to the outside (e.g., a user), and may include one or more of a display, speaker, etc. Optionally, the input device 106 and the output device 108 can be integrated together and implemented using the same interactive device (e.g., a touch screen).

[0041] The image acquisition device 110 can acquire images and store the acquired images in the storage device 104 for use by other components. The image acquisition device 110 can be a standalone camera or a camera in a mobile terminal, etc. It should be understood that the image acquisition device 110 is only an example, and the electronic device 100 may not include the image acquisition device 110. In this case, other devices with image acquisition capabilities can be used to acquire images and send the acquired images to the electronic device 100.

[0042] For example, an example electronic device for implementing the unlocking method and apparatus according to embodiments of this application may be implemented on a device such as a personal computer or a remote server.

[0043] Below, we will refer to Figure 2 This application describes an unlocking method according to an embodiment of the present application. Figure 2 A schematic flowchart illustrating an unlocking method 200 according to one embodiment of this application is shown. Figure 2 As shown, unlocking method 200 includes steps S210, S220, S230 and S240.

[0044] In step S210, the current unlock image corresponding to the current unlock operation is obtained.

[0045] The unlocking images described herein (including the current unlocking image and previous unlocking images) can be static images or any video frame from a dynamic video. The unlocking image can be the original image acquired by an image acquisition device, or it can be an image obtained after preprocessing the original image (such as digitization, normalization, smoothing, etc.). The aforementioned image acquisition device can be implemented using a conventional image sensor such as a camera, or it can be implemented using a different type of sensor than a camera, such as a thermal fingerprint sensor. It should be noted that preprocessing the original image can include extracting a sub-image containing the target object from the original image acquired by the image acquisition device to obtain the unlocking image. The target object can be any object, including but not limited to: a person or part of the human body, such as a face, fingerprint, palm print, lip print, iris, etc. For example, the unlocking image can be a fingerprint image containing one or more fingerprints of a human body.

[0046] In step S220, at least one previous unlock image corresponding to at least one previous unlock operation prior to the current unlock operation is obtained.

[0047] Either the current unlock operation or the previous unlock operation can be an operation including steps S210 to S240. The current unlock operation is performed for the currently unlocked image, and a previous unlock operation is performed for each previously unlocked image.

[0048] The number of at least one previously unlocked image can be set to any suitable number as needed. For example, at least one previously unlocked image can be N previously unlocked images stored in a storage unit. N can be a preset value, which is a positive integer greater than or equal to 1. For example, if N or more previously unlocked operations have been performed before the current unlocked operation, N of all previously unlocked images can be determined as at least one previously unlocked image. If no N previously unlocked operations have been performed before the current unlocked operation, all previously unlocked images can be determined as at least one previously unlocked image.

[0049] In step S220, at least one previous unlocking operation can be adjacent to each other, i.e., consecutive unlocking operations, or it can be non-consecutive unlocking operations. For example, an unlocking operation can be performed once based on the current unlocking image. After determining whether unlocking is possible, the current unlocking image can be stored in a storage unit, along with its corresponding unlocking result. Thus, when a new unlocking operation is performed next time, the previously stored unlocking image can be used as a previous unlocking image to assist in determining whether unlocking is possible. Alternatively, the unlocking images can be filtered, storing only those with higher quality and discarding those with lower quality. When a new unlocking operation is performed next time, the previously stored high-quality unlocking image is used for auxiliary determination. In this case, at least one previous unlocking operation can be non-consecutive.

[0050] In one example, in step S220, the previous unlock image corresponding to the previous unlock operation adjacent to the current unlock operation can be obtained. That is, only the previous unlock image corresponding to the previous unlock operation closest to the current unlock operation can be obtained each time, and then the current unlock operation can be auxiliaryly judged based on the previous unlock image.

[0051] In step S230, a previously unlocked image is found from at least one previously unlocked image whose first similarity to the current unlocked image is greater than a first similarity threshold.

[0052] In the first example, for the current unlock operation, whether the current unlock is successful can be determined solely based on previously unlocked images. In this case, step S230 can be used to find multiple previously unlocked images whose first similarity to the current unlocked image is greater than a first similarity threshold. If none of the aforementioned multiple previously unlocked images can be found, the current unlock can be optionally determined directly based on the current comparison result, or the current unlock can be explicitly determined to have failed. In the second example, for the current unlock operation, whether the current unlock is successful can be determined by combining the current unlocked image and previously unlocked images. In this case, step S230 can be used to find one or more previously unlocked images whose first similarity to the current unlocked image is greater than a first similarity threshold. If none of the aforementioned one or more previously unlocked images can be found, the current unlock can be optionally determined directly based on the current comparison result, or the current unlock can be explicitly determined to have failed.

[0053] The multiple previously unlocked images found in the first example and one or more previously unlocked images found in the second example may be partially the same, all the same, or all different.

[0054] The number of previously unlocked images searched in the first example and the number of one or more previously unlocked images searched in the second example can be set to any suitable number as needed. In one example, the number of previously unlocked images searched in the first example or the number of one or more previously unlocked images searched in the second example is preset and fixed. For example, the previously unlocked images searched in the first example may be M1 previously unlocked images selected from the above N previously unlocked images, and the one or more previously unlocked images searched in the second example may be M2 previously unlocked images selected from the above N previously unlocked images. M1 is a positive integer greater than or equal to 2 and less than or equal to N. For example, M1 may be 2, 3, 4, etc. M2 is a positive integer greater than or equal to 1 and less than or equal to N. For example, M2 may be 1, 2, 3, etc. In this case, if at least M1 or more of the previously unlocked images have a similarity (referred to as the first similarity in this paper for distinction) greater than the first similarity threshold with the current unlocked image, then M1 of these previously unlocked images can be found as the multiple previously unlocked images involved in the first example. If none of the at least M1 previously unlocked images have a first similarity greater than the first similarity threshold with the current unlocked image, then it can be considered that the required multiple previously unlocked images cannot be found. The method for finding one or more previously unlocked images involved in the second example is similar to the method for finding multiple previously unlocked images involved in the first example, and will not be repeated here.

[0055] In another example, the number of previously unlocked images involved in the first example or the number of one or more previously unlocked images involved in the second example can be determined by the number of previously unlocked images among at least one that have a first similarity greater than a first similarity threshold with the current unlocked image. That is, the number of previously unlocked images involved in the first example or the number of one or more previously unlocked images involved in the second example is equal to the number of previously unlocked images corresponding to a first similarity greater than the first similarity threshold. Of course, if at most one of the at least one previously unlocked image has a first similarity greater than the first similarity threshold with the current unlocked image, the requirement for the number of previously unlocked images in the first example cannot be met.

[0056] In this paper, the similarity between two images can be calculated using any suitable existing or future similarity calculation method. For example, features of the two images can be extracted separately using a neural network, and the similarity between the extracted features can be calculated to represent the similarity between the two images.

[0057] In step S240, based on the previous unlocking results corresponding to each of the multiple previously unlocked images found, or based on the current comparison result and the previous unlocking results corresponding to one or more previously unlocked images found, it is determined whether to unlock. The current comparison result is the comparison result obtained by comparing the current unlocked image with the images in the target database.

[0058] The images in the target database can be template images of authorized users of electronic devices. If the current unlock image matches a database image, the current user is considered an authorized user, and unlocking can proceed.

[0059] When applying unlocking method 200 to the field of fingerprint unlocking, the target database can be a database corresponding to a single finger, meaning the target database only contains fingerprint images corresponding to a single finger. When the number of images in the database is large, classifying and storing them by finger facilitates subsequent comparison, thereby improving comparison efficiency. Optionally, the target database can also be a database corresponding to multiple fingers, meaning the target database can simultaneously contain fingerprint images corresponding to multiple fingers.

[0060] The target database can contain one or more images. If the target database consists of a single image, the current unlock image can be directly compared to it. If the target database contains multiple images, the current unlock image can be compared one by one with each of the multiple database images. The comparison method may include calculating the similarity (referred to as second similarity) between the current unlock image and the database images. The current comparison result may include the second similarity between the current unlock image and each of the aforementioned one or more database images.

[0061] One or more previously unlocked images are images whose first similarity to the current unlocked image is greater than a first similarity threshold. In other words, these previously unlocked images and the current unlocked image can be considered to represent the biometric features of the same person, such as the same person's face, fingerprints, iris, etc. Therefore, the previous unlocking results of multiple previously unlocked images can be combined with each other, or the previous unlocking results of previously unlocked images can be combined with the current comparison results of the current unlocked image. That is, the current unlocking result can be determined by combining historical unlocking information.

[0062] Using the above-described unlocking method, under the current unlocking operation, it can determine whether to unlock based on the unlocking information of multiple previously unlocked images similar to the current unlocked image; or, in addition to considering the comparison results between the current unlocked image and the base database images, it can also combine the unlocking information of one or more previously unlocked images similar to the current unlocked image to comprehensively determine whether to unlock. That is, this scheme can make unlocking judgments based on information from multiple frames of unlocked images rather than information from a single frame of unlocked images, so that unlocking can rely on more image information. Therefore, this approach helps to improve the accuracy of biometric recognition, thereby helping to improve the success rate of unlocking. Exemplarily, the unlocking method according to the embodiments of this application can be implemented in a device, apparatus, or system having a memory and a processor.

[0063] The unlocking method according to the embodiments of this application can be deployed at the image acquisition end, for example, at a personal terminal or a server.

[0064] Alternatively, the unlocking method according to the embodiments of this application can also be deployed distributedly on the server (or cloud) and at personal terminals. For example, an unlocking image can be captured on the client side, and the client side can transmit the captured unlocking image to the server (or cloud) side, whereby the server (or cloud) side performs the unlocking determination.

[0065] In one example, in step S240, the currently unlocked image can be compared with images in the target database to obtain a current comparison result. Subsequently, based on the current comparison result and the previous unlock results corresponding to one or more previously unlocked images, it can be determined whether to unlock.

[0066] In this example, the comparison results between the current unlock image and the base image, as well as the unlock information of the previous unlock image, are used to make a joint judgment. This can improve the success rate of unlocking and thus improve the unlocking performance in application scenarios such as side fingerprint recognition.

[0067] According to an embodiment of this application, the current comparison result includes a second similarity between the currently unlocked image and each of one or more images in the target database. Determining whether to unlock based on the current comparison result and the previous unlock results corresponding to one or more previously unlocked images can include: determining a current unlock threshold based on the previous unlock results corresponding to one or more previously unlocked images; if there is a second similarity greater than the current unlock threshold in the current comparison result, then unlocking is determined to be successful; if there is no second similarity greater than the current unlock threshold in the current comparison result, then unlocking is determined to be unsuccessful.

[0068] The unlocking results of previously unlocked images can be used to determine the current unlocking threshold. The current unlocking threshold is a threshold used to determine whether the currently unlocked image matches an image in the base database. For example, and not limitingly, if there exists a second similarity between any previously unlocked image and the currently unlocked image greater than the current unlocking threshold, the image can be unlocked; otherwise, it can remain unlocked.

[0069] For example, determining the current unlock threshold based on the previous unlock results corresponding to one or more previously unlocked images found includes: directly determining the current unlock threshold based on the previous unlock results corresponding to one or more previously unlocked images found.

[0070] For example, determining the current unlock threshold based on the previous unlock results corresponding to one or more previously unlocked images found includes: determining auxiliary unlock information based on the previous unlock results corresponding to one or more previously unlocked images found; and determining the current unlock threshold based on the auxiliary unlock information.

[0071] The previous unlocking results corresponding to one or more previously unlocked images can be combined to determine the auxiliary unlocking information, and then the current unlocking threshold can be determined based on this information. The combination method can be set as needed; for example, the auxiliary unlocking information can be used to determine whether to unlock or not by statistically analyzing the number or proportion of one or more previous unlocking results indicating successful or failed unlocking. Optionally, the average similarity contained in one or more previous unlocking results can also be calculated to determine the auxiliary unlocking information. These will be described below.

[0072] When comparing with the database, a threshold is typically used to determine whether the corresponding unlocked image matches an image in the database, i.e., whether it belongs to the database and can be unlocked. Therefore, determining the current unlock threshold using auxiliary unlock information is a simple way to incorporate unlock information from previously unlocked images into the current unlock operation's judgment.

[0073] According to an embodiment of this application, determining the current unlock threshold based on the auxiliary unlock information includes: increasing the initial unlock threshold to obtain the current unlock threshold when the auxiliary unlock information indicates that the device should not be unlocked; and / or, decreasing the initial unlock threshold to obtain the current unlock threshold when the auxiliary unlock information indicates that the device should be unlocked.

[0074] The current unlock operation can have its own initial unlock threshold. For example, and not restrictively, the initial unlock threshold can be a preset, fixed value.

[0075] Optionally, if the assisted unlocking information indicates that unlocking is not allowed, the initial unlocking threshold can be directly set as the current unlocking threshold (i.e., the initial unlocking threshold is not adjusted). Alternatively, if the assisted unlocking information indicates that unlocking is allowed, the initial unlocking threshold can also be directly set as the current unlocking threshold.

[0076] It's important to note that for both the "unlock" and "disable" scenarios indicated by the unlock assistance message, the initial unlock threshold will not be simultaneously set as the current unlock threshold. That is, if the initial unlock threshold is increased to obtain the current unlock threshold when the unlock assistance message indicates "disable," then when the unlock assistance message indicates "disable," you can choose to decrease the initial unlock threshold to obtain the current unlock threshold or set the initial unlock threshold as the current unlock threshold. Conversely, if the initial unlock threshold is decreased to obtain the current unlock threshold when the unlock assistance message indicates "disable," then when the unlock assistance message indicates "disable," you can choose to increase the initial unlock threshold to obtain the current unlock threshold or set the initial unlock threshold as the current unlock threshold.

[0077] If the unlocking information from a previously unlocked image indicates that it can be unlocked, lowering the threshold for comparison with the database makes the current unlocked image more likely to be considered a match for the database and thus unlockable. Conversely, if the unlocking information from a previously unlocked image indicates that it cannot be unlocked, lowering the threshold for comparison with the database makes the current unlocked image more likely to be considered a mismatch for the database and thus unlockable. This allows for faster and more accurate determination of whether the current unlocking operation will succeed by referencing unlocking information from similar previously unlocked images.

[0078] According to an embodiment of this application, determining the auxiliary unlocking information based on the previous unlocking results corresponding to one or more previously unlocked images includes: if a first preset number or a first preset proportion of the previous unlocking results corresponding to one or more previously unlocked images are successfully unlocked, then the auxiliary unlocking information is determined to be unlocked; otherwise, the auxiliary unlocking information is determined to be not unlocked.

[0079] Both the first preset number and the first preset ratio can be set to any suitable value as needed, and this application does not impose any restrictions on them. For example, it can be set that if at least 50% of the previous unlocking results are successful, the auxiliary unlocking information can be determined as unlocked. As another example, it can be set that if at least one of the previous unlocking results is successful, the auxiliary unlocking information can be determined as unlocked.

[0080] The result of each unlocking operation can include indication information, which can be used to indicate whether the unlocking operation was successful. In this case, it is possible to know whether the result of each unlocking operation was successful or failed, and then determine auxiliary unlocking information based on the success or failure information.

[0081] This embodiment only requires counting the number or proportion of previous unlocking results that indicate successful or unsuccessful unlocking. This approach involves relatively little computation and helps improve unlocking efficiency.

[0082] According to an embodiment of this application, the previous unlocking result includes a second similarity between the corresponding previously unlocked image and the images in the target database. Based on the previous unlocking results corresponding to one or more previously unlocked images, determining the auxiliary unlocking information includes: taking a weighted average of the second similarities corresponding to one or more previously unlocked images to obtain a first average similarity; if the first average similarity is greater than the second similarity threshold, the auxiliary unlocking information is determined as unlocking, otherwise the auxiliary unlocking information is determined as not unlocking.

[0083] The unlocking result of each unlocking operation may also include the comparison result between the corresponding unlocked image and the base database image, which may include a second similarity between the corresponding unlocked image and the base database image.

[0084] When there is only one image in the target database, the second similarity calculated during database comparison is one. When there are multiple images in the target database, the second similarity calculated during database comparison can be multiple. For any unlocked image, the second similarity recorded in its corresponding unlocking result or comparison result can include the second similarity between the unlocked image and each image in the target database, or the second similarity between the unlocked image and similar images in the target database, or the second similarity between the unlocked image and the most similar image in the target database. A similar image is a database image whose second similarity to the corresponding unlocked image is greater than the corresponding unlocking threshold (e.g., the current unlocking threshold corresponding to the current unlocked image). The most similar image is a database image with the highest second similarity to the corresponding unlocked image, or a database image with the highest second similarity to the corresponding unlocked image and whose second similarity is greater than the corresponding unlocking threshold. In other words, the number of second similarities recorded in the unlocking result or comparison result corresponding to any unlocked image can be one or more.

[0085] A preferable approach is to record only the second similarity between the corresponding unlocked image and the most similar image in the target database in the unlocking or comparison results.

[0086] A first average similarity can be obtained by taking a weighted average of the second similarities corresponding to one or more previously unlocked images. The first average similarity is then compared with a similarity threshold (second similarity threshold) to determine whether the auxiliary unlocking information indicates unlocking or not.

[0087] This method of averaging similarity results in a more accurate synthesis of unlocking information from previously unlocked images, which helps improve the accuracy of unlocking judgments for the current unlocked image.

[0088] According to an embodiment of this application, in the weighted average, for any previously unlocked image among one or more previously unlocked images found, the closer the execution time of the previous unlocking operation corresponding to the previously unlocked image is to the execution time of the current unlocking operation, the greater the weight of the second similarity corresponding to the previously unlocked image.

[0089] If the previous unlocking image was acquired and unlocked too long ago, it may not be compatible with the current acquisition environment, leading to a decrease in the reliability and validity of the data. Therefore, theoretically, the closer the current unlocking image is, the more reliable the unlocking information from previous unlocking images. By assigning higher weights to previous unlocking images closer to the current unlocking operation, the focus can be placed on assisting unlocking based on more recent previous unlocking images, which can further improve the accuracy of recognition.

[0090] According to an embodiment of this application, the previous unlocking result includes the previous comparison result. The current comparison result and the previous comparison result each include a second similarity between the corresponding unlocked image and each of the one or more images in the target database. Based on the current comparison result and the previous unlocking results corresponding to the one or more previously unlocked images, determining whether to unlock includes: taking a weighted average of the second similarities corresponding to the one or more previously unlocked images and the current unlocked image to obtain a second average similarity; comparing the second average similarity with a current unlocking threshold; if the second average similarity is greater than the current unlocking threshold, then unlocking is determined to be successful; if the second average similarity is not greater than the current unlocking threshold, then unlocking is determined to be unsuccessful.

[0091] For example, the current unlocking result may include the current comparison result and current indication information, and the previous unlocking result may include the previous comparison result and previous indication information. The current comparison result and the previous comparison result each include a second similarity between the corresponding unlocked image and each image in one or more database images in the target database. The current indication information and the previous indication information are each used to indicate whether the unlocking was successful.

[0092] The above text has already described several cases of the second similarity included in the comparison results, which will not be repeated here.

[0093] For example, the indication information (including current indication information and previous indication information) may include a first indication information for indicating successful unlocking and a second indication information for indicating unlocking failure. For example, and not limitingly, the first indication information and the second indication information may be represented by the numbers 1 and 0, respectively.

[0094] For any unlocked image, the comparison result between it and the base database images can be obtained (i.e., the second similarity), as well as the final information on whether the unlock was successful (i.e., the indication information). When storing the unlocking results, the second similarity and the indication information can be stored together for later use in assisted unlocking.

[0095] In this embodiment, the second similarity scores corresponding to the previously unlocked images and the current unlocked image can be averaged together. For example, assuming that three similar previously unlocked images are found for the current unlocked image, the second similarity scores corresponding to these three previously unlocked images and the current unlocked image can be averaged to obtain the second average similarity score.

[0096] After calculating the second average similarity, it is compared with the current unlock threshold, and a determination is made based on the comparison result as to whether unlocking is currently possible. Optionally, the current unlock threshold here may be the initial unlock threshold described herein.

[0097] In this embodiment, the second similarity score between the previously unlocked image and the database is incorporated into the similarity calculation of the current unlocked image. The combined second similarity score (i.e., the second average similarity score) is used as the similarity result of the current unlocked image. It can be understood that if the previously unlocked image matches the database image, combining their corresponding second similarities may increase the similarity result of the current unlocked image, making it more likely to be considered a match with the database and thus unlockable. Conversely, if the previously unlocked image does not match the database image well, combining their corresponding second similarities may decrease the similarity result of the current unlocked image, making it more likely to be considered a mismatch with the database and thus ununlockable.

[0098] The above method can also achieve the goal of determining the unlocking result of the current unlocking image by referring to the unlocking information of similar previously unlocked images.

[0099] According to the embodiments of this application, in the weighted average, the weight of the second similarity corresponding to the current unlocked image is the largest, and for any previously unlocked image among one or more previously unlocked images found, the closer the execution time of the previous unlocking operation corresponding to the previous unlocked image is to the execution time of the current unlocking operation, the greater the weight of the second similarity corresponding to the previous unlocked image.

[0100] This embodiment can be understood by referring to the weight setting method when calculating the first average similarity above, and will not be repeated here.

[0101] According to an embodiment of this application, the current comparison result includes a second similarity between the current unlocked image and each of one or more images in the target database. After finding a previously unlocked image from at least one previously unlocked image whose first similarity to the current unlocked image is greater than a first similarity threshold (step S230), the method 200 may further include: if no one or more previously unlocked images are found, determining the current unlock threshold as an initial unlock threshold; if there is a second similarity greater than the current unlock threshold in the current comparison result, then determining that the unlocking is successful; if there is no second similarity greater than the current unlock threshold in the current comparison result, then determining that the unlocking has failed.

[0102] If, in step S230, no previously unlocked image with a first similarity greater than the first similarity threshold is found among at least one previously unlocked image, it indicates that there is not much correlation between the current unlocked image and the previously unlocked images. In this case, the initial unlock threshold can be retained without adjustment, and the comparison result between the current unlocked image and the base database image can be directly used to determine whether it can be unlocked.

[0103] The above description, in conjunction with embodiments, describes the process of determining whether to unlock based on the current comparison result and the previous unlock results corresponding to one or more previously unlocked images.

[0104] In one example, in step S240, the decision to unlock can be made based on the previous unlock results corresponding to each of the multiple previously unlocked images. In this case, since there are multiple previously unlocked images, it is not necessary to compare the current unlocked image with images in the target database; instead, the decision to unlock can be made directly based on the previous unlock results corresponding to these multiple previously unlocked images.

[0105] In this example, since there is no need to compare the current unlocked image with the base image, this approach can greatly reduce the workload of the unlocking system and effectively improve the unlocking speed.

[0106] The following describes, with reference to an embodiment, the process of determining whether to unlock based on the previous unlock results corresponding to each of the multiple previously unlocked images found.

[0107] According to embodiments of this application, determining whether to unlock based on the previous unlock results corresponding to each of the found previously unlocked images includes one of the following: if the previous unlock results corresponding to each of the found previously unlocked images are all successful, then unlocking is determined to be successful; otherwise, unlocking is determined to be unsuccessful. If a second preset number or a second preset proportion of the found previously unlocked images have previous unlock results corresponding to successful unlocking, then unlocking is determined to be successful; otherwise, unlocking is determined to be unsuccessful. If a third preset number or a third preset proportion of the found previously unlocked images have previous unlock results corresponding to successful unlocking, then unlocking is determined to be successful; otherwise, unlocking is determined to be unsuccessful. If the previous unlock results corresponding to each of the found previously unlocked images are all successful, and the found previously unlocked images are all successful... If the image is a previously unlocked image acquired within a first preset time period before the current time, then unlocking is determined to be successful; otherwise, unlocking is determined to be unsuccessful. If among the multiple previously unlocked images found, there is a fourth preset number or a fourth preset proportion of previously unlocked images whose previous unlocking result is successful, and the fourth preset number or a fourth preset proportion of previously unlocked images were previously unlocked images acquired within a second preset time period before the current time, then unlocking is determined to be successful; otherwise, unlocking is determined to be unsuccessful. If among the multiple previously unlocked images found, there is a fifth preset number or a fifth preset proportion of consecutive previously unlocked images whose previous unlocking result is successful, and the fifth preset number or a fifth preset proportion of consecutive previously unlocked images were previously unlocked images acquired within a third preset time period before the current time, then unlocking is determined to be successful; otherwise, unlocking is determined to be unsuccessful.

[0108] The preset quantities, preset ratios, or preset time periods mentioned above can all be set to any suitable value as needed, and this application does not impose any restrictions on this.

[0109] When determining whether to unlock based solely on previous unlocking results, multiple methods can be used to determine this. For example, it can be limited to a condition where all previously unlocked images have been successfully unlocked before the current unlock can succeed. Alternatively, it can be limited to a condition where a predetermined number or a predetermined proportion of previously unlocked images have been successfully unlocked before the current unlock can succeed. Yet another example is that it can be limited to a condition where a predetermined number or a predetermined proportion of consecutive previously unlocked images have been successfully unlocked before the current unlock can succeed. According to the above embodiments, the current unlocking decision is determined based on the previous unlocking results corresponding to each of multiple previously unlocked images. This method utilizes the sequence information of multiple preceding and following frames for auxiliary unlocking judgment, avoiding judgment based solely on information from a single unlocked image, and can significantly improve the accuracy and success rate of unlocking.

[0110] Furthermore, for the examples above, the time period of the previously unlocked images used in the judgment can be further limited. For example, it can be limited to all previously unlocked images within the past few minutes, or a preset number or proportion of previously unlocked images, or a preset number or proportion of consecutive previously unlocked images. If all of these images unlock successfully, then the current unlock is considered successful. When the time interval between the current unlocked image and the previous unlocked images is short, if the similarity between the current unlocked image and multiple or consecutive previous unlocked images is greater than a threshold, then it can be considered a continuous operation by the same user. In this case, the unlock can be determined directly based on multiple previous unlock results. This eliminates the need to compare the current unlocked image with images in the database, which helps improve unlocking speed and thus enhances the user experience.

[0111] According to an embodiment of this application, after obtaining the current unlock image corresponding to the current unlock operation (step S210), method 200 may further include: selecting representative images from multiple previous image sets or multiple base libraries that correspond one-to-one with multiple different target objects to obtain representative image sets corresponding to each of the multiple different target objects; comparing the current unlock image with each representative image set to determine a third similarity between the current unlock image and each representative image set; selecting the previous image set and base library corresponding to the representative image set with the largest third similarity; determining the image in the selected previous image set as at least one previous unlock image; and determining the selected base library as the target base library.

[0112] The target object can be any biometric object, including but not limited to: face, fingerprint, palm print, lip print, iris, etc. Different target objects can be target objects in different parts of the same person, such as different fingers of the same person (e.g., index finger and middle finger). Different target objects can also be target objects in the same part of different people, such as the same finger in the same part of different people (e.g., both are index fingers). Of course, different target objects can also be target objects in different parts of different people.

[0113] For example, a database can be established for each of a person's 10 fingers. Each time a current unlock image is acquired, the finger corresponding to the current unlock image can be determined by comparing it with the database. The current unlock image can then be stored in the storage location corresponding to that finger, forming a set of previous images corresponding to that finger.

[0114] When the current unlock image is acquired, it can first be determined which finger it belongs to. This can be achieved by comparing it with the representative image set corresponding to each finger. The representative image set corresponding to any finger can come from the base database corresponding to that finger or the previous image set.

[0115] The finger to which the current unlock image is most similar is identified. After determining which finger the current unlock image belongs to, steps S220 to S240 can be performed based solely on the previous image set and the database corresponding to that finger. This approach eliminates the need to compare all previous unlock images and database images for each finger, significantly reducing computational complexity.

[0116] According to an embodiment of this application, after obtaining the current unlock image corresponding to the current unlock operation (step S210), method 200 may further include: for each of a plurality of different target objects, taking an image in the previous image set corresponding to the target object as at least one previous unlock image, and taking the base library corresponding to the target object as the target base library, and executing steps S220 to S240.

[0117] For example, for multiple fingers, the current unlock image can be compared one by one with the previous unlock image and the corresponding image in the database for each finger to determine whether unlocking is possible. When the number of images to be compared is small, this approach does not require the creation of multiple databases, thus reducing storage requirements.

[0118] According to an embodiment of this application, at least one previously unlocked image is stored in a storage unit, and the number of previously unlocked images stored in the storage unit is fixed.

[0119] The storage unit stores a fixed number of previously unlocked images. Once the number of previously unlocked images stored reaches this fixed number, each time a new unlocked image is obtained, the user can choose whether to update the stored previously unlocked images with the new unlocked image.

[0120] According to an embodiment of this application, the unlocking method may further include: calculating the image quality of previously unlocked images stored in the storage unit; deleting the target previously unlocked image and the corresponding previously unlocked result from the storage unit, and storing the current unlocked image and the current unlocked result corresponding to the current unlocked image in the storage unit, wherein the target previously unlocked image is the image with the lowest image quality among the previously unlocked images stored in the storage unit.

[0121] Optionally, the step of deleting the target previously unlocked image and its corresponding previously unlocked result from the storage unit and storing the current unlocked image and its corresponding current unlocked result in the storage unit is performed if the image quality of the target previously unlocked image is lower than that of the current unlocked image, or is performed directly.

[0122] The worst-quality previous unlock image in the storage unit can be replaced with the current unlock image. Of course, the replaced previous unlock image must have a similarity greater than a first similarity threshold (i.e., both belong to the same target object) with the current unlock image. In one example, replacement is only performed when the current unlock image is of better quality than the worst-quality previous unlock image. In another example, the quality of both images can be disregarded, and the current unlock image can directly replace the worst-quality previous unlock image. Since the current unlock image was acquired more recently, it better represents the user's current and most authentic biometric state. Therefore, replacing the previous unlock image with the current unlock image facilitates timely updates of the unlock images and corresponding unlock results stored in the storage unit, helping to improve the effectiveness and accuracy of unlocking.

[0123] Image quality can be determined using any suitable quality evaluation criteria. For example, image quality can be measured using one or more of the following criteria: image blurriness, the degree of occlusion of objects in the image, the pose of objects in the image, the size of objects in the image, etc. An overall image quality score can be calculated by combining these criteria, and the image quality can be evaluated based on the magnitude of the image quality score.

[0124] By using the above method, relatively fresh or high-quality images and their unlocking results can be stored in the storage unit, which helps to reduce the difficulty of subsequent unlocking and further improve the accuracy of subsequent unlocking.

[0125] According to an embodiment of this application, at least one previously unlocked image is a previously unlocked image acquired within a fourth preset time period prior to the current time.

[0126] The fourth preset time period can be set to any suitable value as needed, and this application does not impose any restrictions on it. At least one previously unlocked image is set to a previously unlocked image within a certain time period before the current moment. If different unlocked images were acquired within a relatively short period of time in the past, the probability of the images containing the same target object is higher, or the users' biometric features are similar. Therefore, using image data within this time period for unlocking is beneficial to improving the accuracy or efficiency of unlocking.

[0127] For example, previously unlocked images and their corresponding previously unlocked results that are more than five preset time periods away from the current time can be deleted from the storage unit, which helps to save the terminal's storage resources. The fifth preset time period can be set to any suitable value as needed, and this application does not limit it.

[0128] According to an embodiment of this application, step S230 may include: aligning the current unlocked image with each of at least one previously unlocked image; determining a specific number of the most similar previously unlocked images based on the alignment results; calculating a first similarity between each of the specific number of previously unlocked images and the current unlocked image, and selecting multiple previously unlocked images with a first similarity greater than a first similarity threshold as multiple previously unlocked images found, or selecting one or more previously unlocked images with a first similarity greater than the first similarity threshold as one or more previously unlocked images found.

[0129] The specific number can be any suitable number, and it can be set as needed. For example, the specific number can be one.

[0130] For each of at least one previously unlocked image, image alignment, or image registration, can be performed between that previously unlocked image and the current unlocked image. The alignment or registration result between the previously unlocked image and the current unlocked image reflects the similarity between them; however, this alignment or registration result is a relatively coarse similarity estimate and does not represent the accurate similarity between the previously unlocked image and the current unlocked image. Subsequently, the previously unlocked image most similar to the current unlocked image can be found based on the alignment or registration result. At this point, the accurate similarity between the most similar previously unlocked image and the current unlocked image is unknown. Later, the similarity can be calculated only between the most similar previously unlocked image and the current unlocked image, and the previously unlocked image with a first similarity greater than a first similarity threshold can be selected. This scheme has a smaller overall computational cost because it does not require calculating the accurate similarity between all previously unlocked images and the current unlocked image.

[0131] According to an embodiment of this application, finding a previously unlocked image from at least one previously unlocked image whose first similarity to the current unlocked image is greater than a first similarity threshold includes: calculating a first similarity between the current unlocked image and each of at least one previously unlocked image; selecting a plurality of previously unlocked images from at least one previously unlocked image whose first similarity is greater than the first similarity threshold as a plurality of previously unlocked images found, or selecting one or more previously unlocked images whose first similarity is greater than the first similarity threshold as one or more previously unlocked images found.

[0132] For each of at least one previously unlocked image, the similarity between that previously unlocked image and the current unlocked image can be directly calculated. Then, a previously unlocked image can be directly selected from these images based on the first similarity score. This approach requires calculating the exact similarity between all previously unlocked images and the current unlocked image, resulting in a relatively high overall computational cost, but it offers high accuracy.

[0133] Figure 3A flowchart illustrating an unlocking method according to an embodiment of this application is shown. Figure 3 As shown, an exemplary process of the unlocking method of this application can be as follows: First, the unlocking information of the first K frames (K is any positive integer greater than or equal to 1) in the unlocking sequence is saved, including the original unlocking image and the unlocking result (unlocked / unlocked or unlocked successfully / unlocked failed) corresponding to the unlocking image; then, in each unlocking comparison, in addition to comparing with the base database images, the most similar image can be selected from the K previously unlocked images saved, and the current unlocking image is compared with this selected image ( Figure 3 The comparison is performed on the nth frame. Finally, 1. If the current unlock image and the previous unlock image are successfully matched, that is, the similarity between the two is greater than the first similarity threshold, it can be considered that there is information coupling between the current unlock image and the previous unlock image. Therefore, auxiliary judgment can be made based on the unlock result of the previous unlock image. For example, if the unlock result of the previous unlock image indicated "unlocked", the current unlock threshold is lowered; otherwise, the current unlock threshold is raised. 2. If the current unlock image and the previous unlock image are not successfully matched (…), the comparison is performed on the nth frame. Figure 3 If the case is not shown, it means that there is no strong correlation between the two. In this case, whether to unlock can be determined solely based on the comparison results between the currently unlocked image and the base image.

[0134] The advantages of the unlocking method of this application are described below with reference to Table 1. Table 1 shows the recognition effect after applying the unlocking method 200 to the side fingerprint unlocking dataset.

[0135] Table 1. Recognition results after applying unlocking method 200 to the side fingerprint unlocking dataset.

[0136] FA FRR@Regular FRR@Rotation Existing methods 23 / 193896 5.22％ 12.45％ This application method 7 / 193896 5.12％ 12.42％

[0137] In Table 1, FA represents the number of different fingers identified as the same finger during fingerprint comparison, and FRR represents the ratio of the same finger identified as different fingers. "FRR@Normal" indicates the FRR when comparing normal fingerprint images. "FRR@Rotation" indicates the FRR when comparing rotated fingerprint images. As can be seen from Table 1, the unlocking method provided in this application embodiment can significantly reduce the number of FAs while maintaining the same FRR. It can be understood that if FA remains constant, the FRR will be significantly increased. FA corresponds to the security of unlocking, while FRR corresponds to the success rate of unlocking. Therefore, under the premise of maintaining the same security, the unlocking method of this application will have a higher unlocking success rate.

[0138] According to another aspect of this application, an unlocking device is provided. Figure 4A schematic block diagram of an unlocking device 400 according to one embodiment of this application is shown.

[0139] like Figure 4 As shown, the unlocking device 400 according to an embodiment of this application includes a first acquisition module 410, a second acquisition module 420, a search module 430, and a determination module 440. Each module can respectively perform the functions described above in conjunction with... Figure 2-3 The following describes the steps / functions of the unlocking method. Only the main functions of each component of the unlocking device 400 are described below, while the details already described above are omitted.

[0140] The first acquisition module 410 is used to acquire the current unlock image corresponding to the current unlock operation. The first acquisition module 410 can be composed of... Figure 1 The processor 102 in the illustrated electronic device executes program instructions stored in the storage device 104 to achieve this.

[0141] The second acquisition module 420 is used to acquire at least one previously unlocked image corresponding to at least one previous unlocked operation prior to the current unlocked operation. The second acquisition module 420 may be composed of... Figure 1 The processor 102 in the illustrated electronic device executes program instructions stored in the storage device 104 to achieve this.

[0142] The search module 430 is used to search for, from at least one previously unlocked image, a previously unlocked image whose first similarity to the currently unlocked image is greater than a first similarity threshold. The search module 430 may be composed of... Figure 1 The processor 102 in the illustrated electronic device executes program instructions stored in the storage device 104 to achieve this.

[0143] The determining module 440 is used to determine whether to unlock based on the previous unlocking results corresponding to each of the multiple previously unlocked images found, or based on the current comparison result and the previous unlocking results corresponding to one or more previously unlocked images found. The current comparison result is the comparison result obtained by comparing the current unlocked image with images in the target database. The determining module 440 can be composed of... Figure 1 The processor 102 in the illustrated electronic device executes program instructions stored in the storage device 104 to achieve this.

[0144] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0145] Figure 5 A schematic block diagram of an electronic device 500 according to an embodiment of this application is shown. The electronic device 500 includes a storage device (i.e., a memory) 510 and a processor 520.

[0146] Storage device 510 stores computer program instructions for implementing corresponding steps in the unlocking method according to embodiments of this application.

[0147] The processor 520 is used to run computer program instructions stored in the storage device 510 to perform the corresponding steps of the unlocking method according to the embodiments of this application.

[0148] In one embodiment, computer program instructions executed by processor 520 are used to perform the following steps: acquiring a current unlock image corresponding to the current unlock operation; acquiring at least one previous unlock image corresponding to at least one previous unlock operation prior to the current unlock operation; searching for a previous unlock image from the at least one previous unlock image whose first similarity to the current unlock image is greater than a first similarity threshold; determining whether to unlock based on the previous unlock results corresponding to each of the found previous unlock images, or based on the current comparison result and the previous unlock results corresponding to each of the found one or more previous unlock images, wherein the current comparison result is a comparison result obtained by comparing the current unlock image with images in the target database.

[0149] For example, the electronic device 500 may also include an image acquisition device 530. The image acquisition device 530 is used to acquire an unlock image. The image acquisition device 530 is optional, and the electronic device 500 may not include the image acquisition device 530. In this case, other image acquisition devices can be used to acquire the unlock image and send the acquired unlock image to the electronic device 500.

[0150] Furthermore, according to embodiments of this application, a storage medium is also provided, on which program instructions are stored. When the program instructions are run by a computer or processor, they are used to execute corresponding steps of the unlocking method of this application embodiment and to implement corresponding modules in the unlocking device according to embodiments of this application. The storage medium may, for example, include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media.

[0151] In one embodiment, when the program instructions are executed by a computer or processor, the computer or processor may implement the various functional modules of the unlocking device according to the embodiments of this application, and / or may execute the unlocking method according to the embodiments of this application.

[0152] In one embodiment, the program instructions are used to perform the following steps at runtime: obtaining a current unlock image corresponding to the current unlock operation; obtaining at least one previous unlock image corresponding to at least one previous unlock operation before the current unlock operation; searching for a previous unlock image from the at least one previous unlock image whose first similarity to the current unlock image is greater than a first similarity threshold; determining whether to unlock based on the previous unlock results corresponding to each of the found previous unlock images, or based on the current comparison result and the previous unlock results corresponding to each of the found one or more previous unlock images, wherein the current comparison result is the comparison result obtained by comparing the current unlock image with images in the target database.

[0153] Furthermore, according to an embodiment of this application, a computer program product is also provided, the computer program product including a computer program, which is used to execute the above-described unlocking method 200 when running.

[0154] Each module in the electronic device according to the embodiments of this application can be implemented by the processor of the electronic device unlocked according to the embodiments of this application running computer program instructions stored in the memory, or by computer instructions stored in a computer-readable storage medium of the computer program product according to the embodiments of this application being executed by a computer.

[0155] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of this application. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of this application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.

[0156] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0157] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0158] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0159] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more aspects of the various applications, features of this application are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of this application. However, this approach should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, the inventive point lies in solving the corresponding technical problem with fewer features than all features of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0160] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0161] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0162] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the unlocking device according to the embodiments of this application. This application can also be implemented as a device program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0163] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0164] The above are merely specific embodiments or descriptions of specific embodiments of this application. The scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. The scope of protection of this application shall be determined by the scope of the claims.

Claims

1. An unlocking method, comprising: Get the current unlock image corresponding to the current unlock operation; Obtain at least one previously unlocked image corresponding to at least one previous unlocked operation prior to the current unlocked operation; Find, from the at least one previously unlocked image, a previously unlocked image whose first similarity to the current unlocked image is greater than a first similarity threshold; When there are multiple previously unlocked images, the determination of whether to unlock is based on the previous unlocking results corresponding to each of the multiple previously unlocked images found. Alternatively, based on the current comparison result and the previous unlock results corresponding to one or more previously unlocked images found, determine whether to unlock; Wherein, the current comparison result is the comparison result obtained by comparing the current unlocked image with the base images in the target base database; the unlocking result of the previous unlocked image is used to determine the current unlocking threshold, which is a judgment threshold for judging the comparison result of the current unlocked image and the base images; or, the previous unlocking result includes the previous comparison result, and the current comparison result and the previous comparison result each include the second similarity between the corresponding unlocked image and each base image in one or more base images in the target base database, and the second average similarity obtained by weighted averaging the second similarities corresponding to the one or more previous unlocked images and the current unlocked image is used to determine whether to unlock.

2. The method as described in claim 1, wherein, The current comparison result includes a second similarity between the currently unlocked image and each of one or more images in the target database. The step of determining whether to unlock based on the current comparison result and the previous unlock results corresponding to one or more previously unlocked images includes: Based on the previous unlocking results corresponding to each of the one or more previously unlocked images found, the current unlocking threshold is determined; If the current comparison result contains a second similarity greater than the current unlocking threshold, then the unlocking is determined to be successful; If there is no second similarity greater than the current unlock threshold in the current comparison results, then unlocking is determined to have failed.

3. The method as described in claim 2, wherein, The step of determining the current unlock threshold based on the previous unlock results corresponding to each of the one or more previously unlocked images found includes: Based on the previous unlocking results corresponding to one or more previously unlocked images found, auxiliary unlocking information is determined; The current unlocking threshold is determined based on the auxiliary unlocking information.

4. The method of claim 3, wherein, Determining the current unlock threshold based on the assisted unlocking information includes: If the assisted unlocking information indicates no unlocking, the initial unlocking threshold is increased to obtain the current unlocking threshold; and / or, When the assisted unlocking information indicates that unlocking is required, the initial unlocking threshold is lowered to obtain the current unlocking threshold.

5. The method as described in claim 3 or 4, wherein, The determination of auxiliary unlocking information based on the previous unlocking results corresponding to each of the one or more previously unlocked images found includes: If a first preset number or a first preset proportion of the previously unlocked results corresponding to the one or more previously unlocked images found are successfully unlocked, then the auxiliary unlocking information is determined to be unlocked; otherwise, the auxiliary unlocking information is determined to be not unlocked.

6. The method as described in claim 3 or 4, wherein, The previous unlocking results include a second similarity between the corresponding previously unlocked image and the images in the target database. The determination of auxiliary unlocking information based on the previous unlocking results corresponding to each of the found one or more previously unlocked images includes: The weighted average of the second similarity scores corresponding to the one or more previously unlocked images found is used to obtain a first average similarity score. If the first average similarity is greater than the second similarity threshold, the auxiliary unlocking information is determined to be unlocked; otherwise, the auxiliary unlocking information is determined to be locked.

7. The method of claim 6, wherein, In the weighted average, for any one of the found previously unlocked images, the closer the execution time of the previous unlocking operation corresponding to the previously unlocked image is to the execution time of the current unlocking operation, the greater the weight of the second similarity corresponding to the previously unlocked image.

8. The method of claim 1, wherein determining whether to unlock based on the current comparison result and the previous unlock results corresponding to one or more previously unlocked images found includes: The second average similarity is obtained by weighting the second similarity of the one or more previously unlocked images and the current unlocked image. Compare the second average similarity with the current unlock threshold; If the second average similarity is greater than the current unlock threshold, then the unlock is determined to be successful; If the second average similarity is not greater than the current unlocking threshold, then unlocking is determined to have failed.

9. The method of claim 8, wherein, In the weighted average, the second similarity corresponding to the current unlocked image has the largest weight, and for any of the one or more previously unlocked images found, the closer the execution time of the previous unlocking operation corresponding to the previous unlocked image is to the execution time of the current unlocking operation, the greater the weight of the second similarity corresponding to the previous unlocked image.

10. The method according to any one of claims 1 to 4, wherein, The current comparison result includes a second similarity between the currently unlocked image and each of one or more images in the target database. After finding a previous unlocked image from the at least one previously unlocked image whose first similarity to the current unlocked image is greater than a first similarity threshold, the method further includes: If no previously unlocked images are found, the current unlock threshold is determined as the initial unlock threshold; If the current comparison result contains a second similarity greater than the current unlocking threshold, then the unlocking is determined to be successful; If there is no second similarity greater than the current unlock threshold in the current comparison results, then unlocking is determined to have failed.

11. The method of claim 1, wherein, The determination of whether to unlock based on the previous unlock results corresponding to each of the multiple previously unlocked images found includes one of the following: If each of the previously unlocked images found has a previously successful unlock result, then the unlock is determined to be successful; otherwise, the unlock is determined to be unsuccessful. If among the found multiple previously unlocked images, there is a second preset number or a second preset proportion of previously unlocked images whose previous unlocking result is successful, then the unlocking is determined to be successful; otherwise, the unlocking is determined to be unsuccessful. If among the found multiple previously unlocked images, there is a third preset number or a third preset ratio of consecutive previously unlocked images whose previous unlocking results are successful, then the unlocking is determined to be successful; otherwise, the unlocking is determined to be unsuccessful. If each of the previously unlocked images found has a previously successful unlock result, and the previously unlocked images found were previously unlocked images acquired within a first preset time period before the current time, then the unlock is determined to be successful; otherwise, the unlock is determined to be unsuccessful. If among the found multiple previously unlocked images, there is a fourth preset number or a fourth preset proportion of previously unlocked images whose previous unlocking result is successful, and the fourth preset number or a fourth preset proportion of previously unlocked images are previously unlocked images obtained within a second preset time period before the current time, then the unlocking is determined to be successful; otherwise, the unlocking is determined to be unsuccessful. If among the found multiple previously unlocked images, there is a fifth preset number or a fifth preset proportion of consecutive previously unlocked images whose previous unlocking result is successful, and the fifth preset number or a fifth preset proportion of consecutive previously unlocked images are previously unlocked images acquired within a third preset time period before the current time, then the unlocking is determined to be successful; otherwise, the unlocking is determined to be unsuccessful.

12. The method according to any one of claims 1 to 4, 7 to 9, and 11, wherein, After obtaining the current unlock image corresponding to the current unlock operation, the method further includes: Representative images are selected from multiple previous image sets or multiple base libraries that correspond one-to-one with multiple different target objects to obtain representative image sets corresponding to each of the multiple different target objects. The current unlocked image is compared with each representative image set to determine a third similarity between the current unlocked image and each representative image set; Select the previous image set and the base library corresponding to the third image set with the highest similarity; The image in the selected set of previous images is identified as the at least one previously unlocked image; The selected base library is designated as the target base library.

13. The method according to any one of claims 1 to 4, 7 to 9, and 11, wherein, The at least one previously unlocked image is stored in a storage unit, and the number of previously unlocked images stored in the storage unit is fixed.

14. The method of claim 13, wherein, The method further includes: Determine the image quality of the previously unlocked image stored in the storage unit; The target previously unlocked image and its corresponding previously unlocked result are deleted from the storage unit, and the current unlocked image and its corresponding current unlocked result are stored in the storage unit, wherein the target previously unlocked image is the image with the lowest image quality among the previously unlocked images stored in the storage unit.

15. The method according to any one of claims 1 to 4, 7 to 9, 11, and 14, wherein, The at least one previously unlocked image is a previously unlocked image acquired within a fourth preset time period prior to the current moment.

16. An electronic device comprising a processor and a memory, characterized in that, The memory stores computer program instructions, which, when executed by the processor, are used to perform the unlocking method as described in any one of claims 1 to 15.

17. A storage medium on which program instructions are stored, characterized in that, The program instructions are used to execute the unlocking method as described in any one of claims 1 to 15 when the program is run.

18. A computer program product, the computer program product comprising a computer program, characterized in that, The computer program, when running, is used to perform the unlocking method as described in any one of claims 1 to 15.

Images