Fingerprint unlocking time determination method and device, terminal and storage medium

Abstract

The present disclosure relates to a method and device for determining a fingerprint unlocking time, a terminal and a storage medium. The method for determining a fingerprint unlocking time comprises: determining an identification area of a display screen of the terminal, the identification area comprising a fingerprint unlocking area; determining an initial data frame of an unlocking process corresponding to the fingerprint unlocking area; determining an end data frame of the unlocking process corresponding to the fingerprint unlocking area; and determining the unlocking time according to the initial data frame and the end data frame. The method of the present disclosure can realize the automation of optical fingerprint unlocking time determination, improve the efficiency of fingerprint unlocking time testing, save manpower and resources, and shorten the development cycle of terminal products.

Description

Technical Field

[0001] This disclosure relates to the field of automated detection technology, and in particular to a method, apparatus, terminal and storage medium for determining fingerprint unlocking time. Background Technology

[0002] With the widespread application of under-display optical fingerprint technology in terminals, the unlocking time of optical fingerprints is a crucial factor affecting user experience. In the early stages of terminal development, it's necessary to evaluate the unlocking performance of optical fingerprint sensors. Researchers typically record the fingerprint unlocking process using a high frame rate camera, analyze the video frame by frame using specialized software, and then manually identify the start and end times of the fingerprint unlocking process. The unlocking time is calculated based on the frame number difference between the start and end frames and the camera's frame rate. This method of manually determining fingerprint unlocking time is labor-intensive, time-consuming, and inefficient, making it difficult to provide timely test reports to researchers. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a method, apparatus, terminal and storage medium for determining fingerprint unlocking time.

[0004] According to a first aspect of the present disclosure, a method for determining fingerprint unlocking time is provided, applied to a terminal, the method comprising:

[0005] The identification area of ​​the terminal's display screen is determined, and the identification area includes a fingerprint unlocking area;

[0006] Determine the initial data frame of the unlocking process corresponding to the fingerprint unlocking area;

[0007] Determine the end data frame of the unlocking process corresponding to the fingerprint unlocking area;

[0008] The unlocking time is determined based on the initial data frame and the end data frame.

[0009] In an exemplary embodiment, determining the initial data frame of the unlocking process corresponding to the fingerprint unlocking area includes:

[0010] The initial data frame is the first frame whose brightness value in the fingerprint unlocking area is greater than or equal to a first preset threshold during the unlocking process.

[0011] In an exemplary embodiment, determining the end data frame of the unlocking process corresponding to the fingerprint unlocking area includes:

[0012] According to the refresh method of the display screen, an initial refresh area is determined in the identification area, and the size of the initial refresh area is 1 / n of the size of the identification area; where n is the number of data frames that refresh the display screen once.

[0013] The first data frame that makes the preset parameters of the initial refresh area meet the preset conditions will be used as the reference data frame.

[0014] The end data frame is determined based on the reference data frame.

[0015] In an exemplary embodiment, determining the initial refresh area in the identification area according to the refresh mode of the display screen includes:

[0016] When the refresh mode of the display screen is to refresh from the first position to the second position along a preset direction, the starting position of the first position is taken as the starting position of the initial refresh area to determine the initial refresh area.

[0017] In an exemplary embodiment, the step of using the data frame that makes the preset parameters of the initial refresh area satisfy the preset conditions as the reference data frame includes:

[0018] Based on the displayed image after the display screen is unlocked, determine the color-related parameters of the initial refresh area;

[0019] The first data frame when the color-related parameters of the initial refresh area meet the preset conditions is used as the reference data frame.

[0020] In an exemplary embodiment, determining the end data frame based on the reference data frame includes:

[0021] The end data frame is determined based on the reference data frame and the number of data frames that refresh the display screen once.

[0022] In an exemplary embodiment, determining the unlocking time based on the initial data frame and the end data frame includes:

[0023] The total number of frames included in the unlocking process is determined based on the initial data frame and the ending data frame.

[0024] The unlocking time is determined based on the total number of frames included in the unlocking process and the frame rate at which the data frames are acquired.

[0025] In an exemplary embodiment, determining the recognition area of ​​the terminal's display screen includes:

[0026] Based on the rectangle detection algorithm, the vertex coordinates of the display screen in the first data frame are determined;

[0027] Based on the vertex coordinates, the recognition area of ​​the display screen in each data frame is determined.

[0028] In one exemplary embodiment, the method further includes:

[0029] All data frames in the unlocking process are converted from RGB images to HSV images.

[0030] In an exemplary embodiment, the step of using an initial frame of data where the brightness value of the fingerprint unlocking area is greater than or equal to a first preset threshold as the initial data frame includes:

[0031] Determine the sum of the V values ​​of all pixels in the fingerprint unlocking area;

[0032] Obtain the initial frame data frame when the sum of the V values ​​of all pixels in the fingerprint unlocking area is greater than or equal to the second preset threshold;

[0033] The initial data frame is used as the initial data frame.

[0034] In one exemplary embodiment, the displayed image is a preset displayed image;

[0035] The step of using the first frame of data when the color-related parameters of the initial refresh area satisfy the preset condition as the reference data frame includes:

[0036] Obtain the H values ​​of all pixels in the initial refresh area of ​​the displayed image after the display screen is unlocked;

[0037] The first frame of data when the sum of the H values ​​of all pixels in the initial refresh area equals a preset H value is used as the reference data frame.

[0038] In one exemplary embodiment, the method further includes:

[0039] Based on the Hough circle detection algorithm, the center and radius of the circle in the recognition area of ​​the display screen are detected;

[0040] Using the center of the circle as the center of the fingerprint unlocking area, a circle with a radius within a preset range is defined as the fingerprint unlocking area.

[0041] According to a second aspect of the present disclosure, a fingerprint unlocking time determining device is provided, applied to a terminal, the determining device comprising:

[0042] The first determining module is configured to determine the recognition area of ​​the terminal's display screen, the recognition area including a fingerprint unlocking area;

[0043] The second determining module is configured to determine the initial data frame of the unlocking process corresponding to the fingerprint unlocking area;

[0044] The third determining module is configured to determine the end data frame of the unlocking process corresponding to the fingerprint unlocking area;

[0045] The fourth determining module is configured to determine the unlocking time based on the initial data frame and the end data frame.

[0046] In one exemplary embodiment, the second determining module is further configured to:

[0047] The initial data frame is the first frame whose brightness value in the fingerprint unlocking area is greater than or equal to a first preset threshold during the unlocking process.

[0048] In one exemplary embodiment, the third determining module is further configured to:

[0049] According to the refresh method of the display screen, an initial refresh area is determined in the identification area, and the size of the initial refresh area is 1 / n of the size of the identification area; where n is the number of data frames that refresh the display screen once.

[0050] The first data frame that makes the preset parameters of the initial refresh area meet the preset conditions will be used as the reference data frame.

[0051] The end data frame is determined based on the reference data frame.

[0052] In one exemplary embodiment, the third determining module is further configured to:

[0053] When the refresh mode of the display screen is to refresh from the first position to the second position along a preset direction, the starting position of the first position is taken as the starting position of the initial refresh area to determine the initial refresh area.

[0054] In one exemplary embodiment, the third determining module is further configured to:

[0055] Based on the displayed image after the display screen is unlocked, determine the color-related parameters of the initial refresh area;

[0056] The first data frame when the color-related parameters of the initial refresh area meet the preset conditions is used as the reference data frame.

[0057] In one exemplary embodiment, the third determining module is further configured to:

[0058] The end data frame is determined based on the reference data frame and the number of data frames that refresh the display screen once.

[0059] In one exemplary embodiment, the fourth determining module is further configured to:

[0060] The total number of frames included in the unlocking process is determined based on the initial data frame and the ending data frame.

[0061] The unlocking time is determined based on the total number of frames included in the unlocking process and the frame rate at which the data frames are acquired.

[0062] In one exemplary embodiment, the first determining module is further configured to:

[0063] Based on the rectangle detection algorithm, the vertex coordinates of the display screen in the first data frame are determined;

[0064] Based on the vertex coordinates, the recognition area of ​​the display screen in each data frame is determined.

[0065] In one exemplary embodiment, the first determining module is further configured to:

[0066] All data frames in the unlocking process are converted from RGB images to HSV images.

[0067] In one exemplary embodiment, the second determining module is further configured to:

[0068] Determine the sum of the V values ​​of all pixels in the fingerprint unlocking area;

[0069] Obtain the initial frame data frame when the sum of the V values ​​of all pixels in the fingerprint unlocking area is greater than or equal to the second preset threshold;

[0070] The initial data frame is used as the initial data frame.

[0071] In one exemplary embodiment, the displayed image is a preset displayed image;

[0072] The third determining module is further configured to:

[0073] Obtain the H values ​​of all pixels in the initial refresh area of ​​the displayed image after the display screen is unlocked;

[0074] The first frame of data when the sum of the H values ​​of all pixels in the initial refresh area equals a preset H value is used as the reference data frame.

[0075] In one exemplary embodiment, the first determining module is further configured to:

[0076] Based on the Hough circle detection algorithm, the center and radius of the circle in the recognition area of ​​the display screen are detected;

[0077] Using the center of the circle as the center of the fingerprint unlocking area, a circle with a radius within a preset range is defined as the fingerprint unlocking area.

[0078] According to a third aspect of the present disclosure, a terminal is provided, comprising:

[0079] processor;

[0080] Memory used to store processor-executable instructions;

[0081] The processor is configured to perform the method as described in any one of the first aspects of the embodiments of this disclosure.

[0082] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of a device, the device is enabled to perform the method as described in any one of the first aspects of the present disclosure.

[0083] The method described above has the following advantages: it can automate the determination of optical fingerprint unlocking time, which helps to improve the efficiency of fingerprint unlocking time testing, saves manpower and resources for testing, and shortens the development cycle of terminal products.

[0084] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0085] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0086] Figure 1 This is a flowchart illustrating a method for determining fingerprint unlocking time according to an exemplary embodiment;

[0087] Figure 2 This is a flowchart illustrating a method for using an initial data frame as the initial data frame when the brightness value of the fingerprint unlocking area is greater than or equal to a first preset threshold during the unlocking process, according to an exemplary embodiment.

[0088] Figure 3 This is a flowchart illustrating a method for determining the end data frame of the unlocking process corresponding to the fingerprint unlocking area in step S103 according to an exemplary embodiment;

[0089] Figure 4 This is a flowchart illustrating a method for using a data frame that satisfies preset conditions for the preset parameters of the initial refresh area as a reference data frame in step S303, according to an exemplary embodiment.

[0090] Figure 5This is a flowchart illustrating a method for using the first frame of data when the color-related parameters of the initial refresh area meet preset conditions as a reference data frame in step S402, according to an exemplary embodiment.

[0091] Figure 6 This is a flowchart illustrating a method for determining the recognition area of ​​a terminal's display screen in step S101 according to an exemplary embodiment;

[0092] Figure 7 This is a flowchart illustrating a method for determining the fingerprint unlocking area in step S101 according to an exemplary embodiment;

[0093] Figure 8 This is a flowchart illustrating a method for determining fingerprint unlocking time according to an exemplary embodiment;

[0094] Figure 9 This is a block diagram illustrating a fingerprint unlocking time determination device according to an exemplary embodiment;

[0095] Figure 10 This is a block diagram illustrating a fingerprint unlocking time determination device according to an exemplary embodiment. Detailed Implementation

[0096] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0097] In an exemplary embodiment of this disclosure, a method for determining fingerprint unlocking time is provided, which is applied to a terminal, including electronic devices such as smartphones and tablets that have optical fingerprint unlocking functions. Figure 1 This is a flowchart illustrating a method for determining fingerprint unlocking time according to an exemplary embodiment, such as... Figure 1 As shown, it includes the following steps:

[0098] Step S101: Determine the recognition area of ​​the terminal's display screen, including the fingerprint unlocking area;

[0099] Step S102: Determine the initial data frame of the unlocking process corresponding to the fingerprint unlocking area;

[0100] Step S103: Determine the end data frame of the unlocking process corresponding to the fingerprint unlocking area;

[0101] Step S104: Determine the unlock time based on the initial data frame and the end data frame.

[0102] In an exemplary embodiment of this disclosure, to overcome the problems in related technologies, a method for determining fingerprint unlocking time is provided. This method involves determining a recognition area on the terminal's display screen, including a fingerprint unlocking area; determining the initial and final data frames of the unlocking process corresponding to the fingerprint unlocking area; and determining the unlocking time based on the initial and final data frames. This method automates optical fingerprint unlocking time detection, improves the efficiency of fingerprint unlocking time testing, saves testing manpower and resources, and shortens the product development cycle.

[0103] In step S101, during the development of the terminal product, performance testing of the optical fingerprint unlocking function is required, i.e., the fingerprint unlocking time needs to be determined so that developers can perform subsequent development processing based on the fingerprint unlocking time. To determine the fingerprint unlocking time, a high-speed camera is used to capture video stream data including the entire fingerprint unlocking process. The captured video stream data is then input into the algorithm used in this disclosure to determine the fingerprint unlocking time. Since capturing the fingerprint unlocking process with a high-speed camera requires manual hand-holding or a fixed camera stand, the captured image includes not only the terminal but also its surrounding environment. Therefore, after obtaining the video stream data including the entire fingerprint unlocking process, it is first necessary to determine the recognition area of ​​the terminal's display screen in the video stream data. The recognition area of ​​the display screen includes the fingerprint unlocking area, i.e., the light spot area of ​​the optical fingerprint. When determining the recognition area of ​​the terminal's display screen, any recognition algorithm can be used to identify the display screen in the video, such as a target recognition algorithm or a rectangle detection algorithm.

[0104] It should be noted that, in order to facilitate accurate identification of the recognition area of ​​the terminal display screen, the image of each data frame in the video stream data can be filtered to remove noise from the image.

[0105] In step S102, since the fingerprint unlocking time is determined by the fingerprint unlocking process, and the state of the fingerprint unlocking area changes during the unlocking process, after determining the recognition area of ​​the terminal's display screen, it is necessary to identify the fingerprint unlocking area within the recognition area. This can be done by recognizing the shape features of the fingerprint unlocking area within the recognition area. The initial data frame of the fingerprint unlocking process is the frame corresponding to the start time of the unlocking process in the video. The initial data frame corresponding to the fingerprint unlocking area can be determined based on the difference between the state of the fingerprint unlocking area at the start of unlocking and the state of the fingerprint unlocking area after the start of unlocking.

[0106] In step S103, the end data frame of the fingerprint unlocking process is the frame corresponding to the end time when the unlocking process is just completed in the video. The end data frame of the unlocking process corresponding to the fingerprint unlocking area can be determined based on the differences in the images displayed in the recognition area of ​​the display screen before and after unlocking. For example, the colors of the displayed images before and after unlocking can be set to different colors; the displayed image before unlocking can be set to a yellow image, and the displayed image after unlocking can be set to a green image. The end data frame of the unlocking process corresponding to the fingerprint unlocking area can be determined based on the color difference of the images displayed in the recognition area of ​​the display screen before and after unlocking.

[0107] It is understandable that steps S102 and S103 have no specific order and can be executed simultaneously.

[0108] In step S104, after determining the initial and final data frames, the number of frames included in the unlocking process can be determined. When acquiring video stream data including the unlocking process, the frame rate of the high-speed camera used is known. Based on the number of frames included in the unlocking process and the frame rate of the high-speed camera, the unlocking time can be calculated. The acquired video stream data includes multiple data frames. To facilitate recording each data frame, each data frame in the video stream data is represented by a frame number. The frame number can be determined according to actual needs. For example, the frame number of the first data frame in the video stream data can be set to 0, the frame number of the second data frame can be set to 1, and so on, setting the frame number of each data frame sequentially. When the frame number of the first data frame in the video stream data is set to 0, the determined initial data frame frame number is recorded as 'a', and the final data frame frame number is recorded as 'b'. Then, the number of frames included in the unlocking process is 'ba'. If the frame rate of the high-speed camera acquiring the video stream data is F, then the unlocking time t is: t = (ba) / F.

[0109] In the exemplary embodiments of this disclosure, after acquiring video stream data including the fingerprint unlocking process, the recognition area of ​​the terminal's display screen is determined. The recognition area includes a fingerprint unlocking area. The initial data frame and the end data frame of the unlocking process corresponding to the fingerprint unlocking area are determined. Based on the initial data frame and the end data frame, the unlocking time can be automatically calculated. This method can automate the determination of optical fingerprint unlocking time, which is beneficial to improving the efficiency of fingerprint unlocking time testing, saving testing manpower and resources, and shortening the development cycle of terminal products.

[0110] In an exemplary embodiment, step S102, determining the initial data frame corresponding to the fingerprint unlocking area in the unlocking process, includes: taking the initial data frame in which the brightness value of the fingerprint unlocking area is greater than or equal to a first preset threshold as the initial data frame.

[0111] To clearly indicate the fingerprint unlock area to the user, a fingerprint spot is typically displayed on the device's screen; this area is the fingerprint unlock area. Before fingerprint unlocking, the fingerprint spot usually illuminates to indicate this area. For example, when the device is in screen-off mode, the fingerprint spot is dark and not visible. When the user's finger touches the fingerprint unlock area, the spot illuminates, initiating the fingerprint unlocking process. Therefore, the brightness value of the fingerprint unlock area can be used to determine whether the fingerprint unlocking process has begun. A first preset threshold for the fingerprint unlock area's brightness value is determined based on the brightness value when the fingerprint spot illuminates. When the brightness value of the fingerprint unlock area is greater than or equal to the preset threshold, it indicates that the fingerprint spot has illuminated, and the fingerprint unlocking process has begun. Therefore, the first frame in the captured video stream where the brightness value of the fingerprint unlock area is greater than or equal to the preset threshold is used as the initial data frame.

[0112] The brightness value of the fingerprint unlocking area can be calculated using the R, G, and B values ​​of each pixel in the RGB image. For example, the brightness value can be calculated using the formula 0.3*R+0.6*G+0.1*B. Alternatively, the RGB image can be converted to an HSV image, and the brightness value can be represented by the V value of each pixel in the HSV image.

[0113] In one implementation, all data frames during the unlocking process are converted from RGB images to HSV images. Figure 2 This is a flowchart illustrating a method for using an initial data frame as the initial data frame when the brightness value of the fingerprint unlocking area is greater than or equal to a first preset threshold during the unlocking process, according to an exemplary embodiment. Figure 2 As shown, it includes the following steps:

[0114] Step S201: Determine the sum of the V values ​​of all pixels in the fingerprint unlocking area;

[0115] Step S202: Obtain the initial frame data frame when the sum of the V values ​​of all pixels in the fingerprint unlocking area is greater than or equal to the second preset threshold.

[0116] Step S203: Use the initial frame data frame as the initial data frame.

[0117] The images corresponding to all data frames in the video stream data can be converted to HSV images using any method capable of converting RGB images to HSV images. The V value of each pixel in the fingerprint unlocking area of ​​each data frame image is obtained, and the sum of the V values ​​of all pixels in the fingerprint unlocking area of ​​each data frame image is calculated. The second preset threshold is an empirical value, pre-stored in the terminal's memory. When the second preset threshold is predetermined, the sum of the V values ​​of all pixels in the fingerprint unlocking area is determined when the fingerprint spot is lit and not obstructed. However, since the finger may cover part of the spot area at the start of the actual fingerprint unlocking process, for example, 80% of the sum of the V values ​​of all pixels in the fingerprint unlocking area can be used as the second preset threshold. When the sum of the V values ​​of all pixels in the fingerprint unlocking area in the acquired video stream data is greater than or equal to the second preset threshold, it indicates that the fingerprint spot has lit up. Since the fingerprint spot is also lit up during the fingerprint recognition process, the initial frame data frame when the sum of the V values ​​of all pixels in the fingerprint unlocking area is greater than or equal to the second preset threshold can represent the start of the fingerprint unlocking process. The initial frame data frame when the sum of the V values ​​of all pixels in the fingerprint unlocking area is greater than or equal to the second preset threshold is used as the initial data frame.

[0118] For example, if the second preset threshold is M, the sum of the V values ​​of all pixels in the fingerprint unlocking area of ​​the images before the m-th frame in the video stream data is less than M, and the sum of the V values ​​of all pixels in the fingerprint unlocking area of ​​the images from the m-th frame to the (m+3)-th frame is greater than M, then the m-th frame is taken as the initial data frame.

[0119] In one exemplary embodiment, Figure 3 This is a flowchart illustrating a method for determining the end data frame of the unlocking process corresponding to the fingerprint unlocking area in step S103 according to an exemplary embodiment, as shown below. Figure 3 As shown, it includes the following steps:

[0120] Step S301: According to the refresh method of the display screen, determine the initial refresh area in the recognition area. The size of the initial refresh area is 1 / n of the size of the recognition area; where n is the number of data frames for refreshing the display screen once.

[0121] Step S302: The first data frame that makes the preset parameters of the initial refresh area meet the preset conditions is used as the reference data frame.

[0122] Step S303: Determine the end data frame based on the reference data frame.

[0123] After fingerprint unlocking, the image displayed on the terminal screen will appear. The terminal screen needs to refresh data to display the image. Let n be the number of data frames required for one screen refresh. Based on the screen's refresh method, an initial refresh area is determined within the recognition area of ​​the screen. The size of the initial refresh area is 1 / n of the recognition area size. In one example, the initial refresh area is determined by the terminal screen's refresh method. When the screen refreshes along a preset direction from a first position to a second position, the starting position of the first position is used as the starting position of the initial refresh area. For example, when the screen refreshes from the upper right corner to the lower left corner, the first position is the upper right corner, and the second position is the lower left corner. The position corresponding to the upper right corner is used as the starting position of the initial refresh area, and 1 / n of the area from the starting position to the lower left corner is the initial refresh area.

[0124] The preset conditions are those that indicate the display image filling after the screen is unlocked. When the preset parameters of the initial refresh area meet the preset conditions, it means that the initial refresh area has been filled with the unlocked display image. Therefore, the first data frame when the preset parameters of the initial refresh area meet the preset conditions indicates that the initial refresh area has just been filled with the unlocked display image. Other areas in the recognition area of ​​the display screen, except for the initial refresh area, are still filled with the display image before unlocking, and none are filled with the unlocked display image. Taking the first data frame when the preset parameters of the initial refresh area meet the preset conditions as the reference data frame, the reference data frame indicates that only the initial refresh area in the recognition area of ​​the display screen is filled with the unlocked display image, while the end data frame indicates that the entire recognition area of ​​the display screen is filled with the unlocked display image. The total number of frames required to refresh the display screen once is from the reference data frame to the end data frame. Therefore, the end data frame can be calculated based on the reference data frame and the total number of frames required to refresh the display screen once.

[0125] In one embodiment, when determining the end data frame based on the reference data frame, the end data frame is determined according to the following formula:

[0126] F n =F i +n-1

[0127] Among them, F n The frame number that indicates the end of the data frame, F i This indicates the frame number of the reference data frame, and n represents the number of data frames that refresh the display screen once.

[0128] In one embodiment, after determining the initial data frame and the end data frame, in step S104, when determining the unlocking time based on the initial data frame and the end data frame, the unlocking time is determined according to the following formula:

[0129] t=(F i +n-1-F m ) / F

[0130] Among them, F m Indicates the frame number of the starting data frame, F i The reference data frame number is represented by n, the number of data frames refreshed once is represented by n, and the frame rate of the data frame is represented by F.

[0131] It is understandable that, in addition to using the first data frame that satisfies the preset conditions for the initial refresh area as the reference data frame, the first data frame that satisfies the preset conditions for any refresh area of ​​size 1 / n can also be used as the reference data frame. The number of data frames refreshed once on the display is n. Starting from the first position, n refresh areas of size 1 / n can be sequentially divided. For example, using the first data frame that satisfies the preset conditions for the Nth refresh area of ​​size 1 / n as the reference data frame, the ending data frame is determined according to the following formula:

[0132] F n =F i +nN

[0133] Among them, F n The frame number that indicates the end of the data frame, F i The frame number of the reference data frame is indicated, n represents the number of data frames that refresh the display screen once, and N represents the Nth refresh area of ​​size 1 / n in the recognition area of ​​the display screen, starting from the first position.

[0134] In one implementation, Figure 4 This is a flowchart illustrating a method for using a data frame that satisfies preset conditions for the initial refresh region as a reference data frame in step S303, as shown in an exemplary embodiment. Figure 4 As shown, it includes the following steps:

[0135] Step S401: Determine the color-related parameters of the initial refresh area based on the displayed image after the display screen is unlocked;

[0136] Step S402: The first data frame when the color-related parameters of the initial refresh area meet the preset conditions is used as the reference data frame.

[0137] When the terminal display screen is filled with the image after fingerprint unlocking, it indicates that fingerprint unlocking is complete. The display image before and after fingerprint unlocking is different; for example, the colors of the display images before and after unlocking are different. The display image before unlocking is set to yellow, and the display image after unlocking is set to green. Therefore, a reference data frame can be determined based on the difference between the display images before and after unlocking. The difference in the display image can be represented by color-related parameters. The color-related parameters of the initial refresh area are determined by a preset condition that indicates the display image of the initial refresh area is the image after unlocking. When the color-related parameters of the initial refresh area meet the preset condition, it indicates that fingerprint unlocking is complete. Therefore, the first data frame when the color-related parameters of the initial refresh area meet the preset condition is taken as the reference data frame. Thus, in the reference data frame, only the initial refresh area of ​​the display screen is filled with the image after unlocking, while other areas are still filled with the image before unlocking.

[0138] The color-related parameters of the initial refresh area can be represented by the R, G, and B values ​​in the RGB image, or by the H value in the corresponding HSV image of the RGB image.

[0139] In one embodiment, all data frames during the unlocking process are converted from RGB images to HSV images, and the display image after the screen is unlocked is a preset display image. Figure 5 This is a flowchart illustrating a method for using the first frame of data when the color-related parameters of the initial refresh area meet preset conditions as a reference data frame in step S402, according to an exemplary embodiment. Figure 5 As shown, it includes the following steps:

[0140] Step S501: Obtain the H values ​​of all pixels in the initial refresh area of ​​the displayed image after the display screen is unlocked;

[0141] Step S502: The first frame data frame when the sum of the H values ​​of all pixels in the initial refresh area is equal to the preset H value is used as the reference data frame.

[0142] The images corresponding to all data frames in the video stream data can be converted to HSV images using any method capable of converting RGB images to HSV images. The H-value of each pixel in the initial refresh area of ​​each data frame image is obtained, and the sum of the H-values ​​of all pixels in the initial refresh area of ​​each data frame image is calculated. When the unlocked display image is a preset display image, the initial refresh area is filled by the preset display image, and the sum of the H-values ​​of all pixels in the initial refresh area is the preset H-value. To facilitate the determination of the preset H-value and improve the testing efficiency of the reference data frame, the preset display image can be a solid color image with high recognizability. When the sum of the H-values ​​of all pixels in the initial refresh area is the preset H-value, it indicates that the initial refresh area has been filled by the preset display image. Since the initial refresh area is filled by the preset display image throughout the entire refresh process after unlocking, the first data frame where the sum of the H-values ​​of all pixels in the initial refresh area is the preset H-value indicates that other areas of the display's recognition area are still filled by the display image before unlocking. This first data frame where the sum of the H-values ​​of all pixels in the initial refresh area is the preset H-value is used as the reference data frame.

[0143] For example, if the preset H value is H, the sum of the H values ​​of all pixels in the initial refresh area of ​​the images before the h-th frame in the video stream data is not H, and the sum of the H values ​​of all pixels in the initial refresh area of ​​the images from the h-th frame to the h+5-th frame is H, then the h-th frame is used as the reference data frame.

[0144] In one exemplary embodiment, Figure 6 This is a flowchart illustrating a method for determining the recognition area of ​​a terminal's display screen in step S101 according to an exemplary embodiment, as shown below. Figure 6 As shown, it includes the following steps:

[0145] Step S601: Based on the rectangle detection algorithm, determine the vertex coordinates of the display screen in the first data frame;

[0146] Step S602: Determine the recognition area of ​​the display screen in each data frame based on the vertex coordinates.

[0147] When the frame number of the first frame in the video stream is set to 0, after acquiring the video stream data including the fingerprint unlocking process, a rectangle detection algorithm is used to identify the display's recognition area and determine the coordinates of its four vertices in the first frame (frame number 0). Alternatively, the display screen in the video stream can be used as the detection target, and a target detection algorithm can be used to identify the display's recognition area and determine the coordinates of its four vertices. Since the entire fingerprint unlocking process is short, and the video stream data is captured from the same location, the location of the display's recognition area is the same in every frame of the video stream data; that is, the relative position of the display's recognition area in the corresponding frame is the same. Therefore, after determining the vertex coordinates of the display screen based on the first frame of the video stream data, the vertex coordinates are used to crop each of the remaining frames in the video stream data to obtain the display's recognition area in each frame. For example, if the vertex coordinates of the display screen are identified as (x1, y1), (x2, y2), (x3, y3), and (x4, y4) in the first data frame, then the recognition area of ​​the display screen in each subsequent data frame can be determined sequentially based on these vertex coordinates. Determining the recognition area of ​​the display screen based on the vertex coordinates of the display screen in the first data frame shortens the time required to determine the recognition area compared to using a rectangle detection algorithm for each data frame.

[0148] It is understandable that since handheld high-speed cameras may cause the first or first few frames of the captured video stream to be blurry, if the recognition area of ​​the display screen cannot be identified in the first frame using a rectangle detection algorithm, then the rectangle detection algorithm is used for the second frame, i.e., the frame number 1, and so on, which can avoid the problem of not being able to determine the recognition area of ​​the display screen.

[0149] In one exemplary embodiment, Figure 7 This is a flowchart illustrating a method for determining the fingerprint unlocking area in step S101 according to an exemplary embodiment, as shown below. Figure 7 As shown, it includes the following steps:

[0150] Step S701: Based on the Hough circle detection algorithm, detect the center and radius of the circle in the recognition area of ​​the display screen;

[0151] Step S702: Define the circle with a radius within a preset range as the fingerprint unlocking area.

[0152] During fingerprint unlocking, most of the fingerprint unlocking area is obscured by the finger, making direct fingerprint recognition impossible. Since the fingerprint unlocking area is typically circular, and the circular animation during the unlocking process unfolds around its center, the Hough circle detection algorithm is used within the display's recognition area based on the fingerprint unlocking area's shape characteristics to detect the center and radius of all possible circles. Because there is only one fingerprint unlocking area on the display, the number of circles used in the Hough circle detection algorithm is limited to one. Based on empirical judgment of the fingerprint unlocking area's size, the maximum and minimum radii are determined. When the radius of the detected circle falls within a preset range formed by the maximum and minimum radii (i.e., the radius is less than or equal to the maximum radius and greater than or equal to the minimum radius), the area containing that circle is identified as the fingerprint unlocking area. If the radius of the detected circle is greater than the maximum radius, it can be reduced to within a preset range through radius shrinkage, and the area of ​​the circle after radius shrinkage is identified as the fingerprint unlocking area. The radius shrinkage value is an empirical value.

[0153] In an exemplary embodiment of this disclosure, a method for determining fingerprint unlocking time is provided, applied to a terminal. Figure 8 This is a flowchart illustrating a method for determining fingerprint unlocking time according to an exemplary embodiment, such as... Figure 8 As shown, it includes the following steps:

[0154] Step S801: Use a high-speed camera with a frame rate of F to capture and acquire video stream data including the fingerprint unlocking process;

[0155] Step S802: Apply a rectangle detection algorithm to the first data frame in the video stream data;

[0156] Step S803: Determine whether the vertex coordinates of the terminal display screen can be detected;

[0157] If yes, proceed to step S805; if no, proceed to step S804.

[0158] Step S804: After discarding the first data frame in the video stream data, execute steps S802 and S803 on the remaining video stream data.

[0159] Step S805: Determine the recognition area of ​​the display screen in each data frame based on the vertex coordinates;

[0160] Step S806: Based on the Hough circle detection algorithm, detect the center and radius of the circle in the recognition area of ​​the display screen;

[0161] Step S807: Define the circle with a radius within a preset range as the fingerprint unlocking area;

[0162] Step S808: Convert all data frames in the unlocking process from RGB images to HSV images;

[0163] Image preprocessing operations such as filtering are performed on the image;

[0164] Step S809: Determine the sum of the V values ​​of all pixels in the fingerprint unlocking area;

[0165] Step S810: Obtain the initial frame data frame when the sum of the V values ​​of all pixels in the fingerprint unlocking area is greater than or equal to the second preset threshold.

[0166] Step S811: Use the initial frame data frame as the initial data frame F m ;

[0167] Step S812: When the refresh mode of the display screen is to refresh from the first position to the second position along a preset direction, the starting position of the first position is taken as the starting position of the initial refresh area, and the initial refresh area is determined.

[0168] The initial refresh area size is 1 / n of the recognition area size, where n is the number of data frames that refresh the display screen once.

[0169] Step S813: Obtain the H values ​​of all pixels in the initial refresh area of ​​the preset display image after the display screen is unlocked;

[0170] Step S814: The first data frame whose sum of the H values ​​of all pixels in the initial refresh area equals the preset H value is used as the reference data frame F. i ;

[0171] Step S815: Determine the end data frame according to the following formula:

[0172] F n =F i +n-1

[0173] Among them, F n The frame number that indicates the end of the data frame, F i This indicates the frame number of the reference data frame, and n represents the number of data frames that refresh the display screen once.

[0174] Step S816: Determine the unlocking time according to the following formula:

[0175] t=(F i +n-1-F m ) / F

[0176] Among them, F m Indicates the frame number of the starting data frame, F i The reference data frame number is represented by n, the number of data frames refreshed once is represented by n, and the frame rate of the data frame is represented by F.

[0177] It is understandable that steps S809-S811 and steps S812-S815 have no specific order and can be executed simultaneously.

[0178] In an exemplary embodiment of this disclosure, a fingerprint unlocking time determination device is provided, applied to a terminal. Figure 9 This is a block diagram illustrating a fingerprint unlocking time determination device according to an exemplary embodiment, such as... Figure 9 As shown, the determining device includes:

[0179] The first determining module 901 is configured to determine the recognition area of ​​the terminal's display screen, the recognition area including a fingerprint unlocking area;

[0180] The second determining module 902 is configured to determine the initial data frame of the unlocking process corresponding to the fingerprint unlocking area;

[0181] The third determining module 903 is configured to determine the end data frame of the unlocking process corresponding to the fingerprint unlocking area;

[0182] The fourth determining module 904 is configured to determine the unlocking time based on the initial data frame and the end data frame.

[0183] In one exemplary embodiment, the second determining module 902 is further configured to:

[0184] The initial data frame is the first frame whose brightness value in the fingerprint unlocking area is greater than or equal to a first preset threshold during the unlocking process.

[0185] In one exemplary embodiment, the third determining module 903 is further configured to:

[0186] According to the refresh method of the display screen, an initial refresh area is determined in the identification area, and the size of the initial refresh area is 1 / n of the size of the identification area; where n is the number of data frames that refresh the display screen once.

[0187] The first data frame that makes the preset parameters of the initial refresh area meet the preset conditions will be used as the reference data frame.

[0188] The end data frame is determined based on the reference data frame.

[0189] In one exemplary embodiment, the third determining module 903 is further configured to:

[0190] When the refresh mode of the display screen is to refresh from the first position to the second position along a preset direction, the starting position of the first position is taken as the starting position of the initial refresh area to determine the initial refresh area.

[0191] In one exemplary embodiment, the third determining module 903 is further configured to:

[0192] Based on the displayed image after the display screen is unlocked, determine the color-related parameters of the initial refresh area;

[0193] The first data frame when the color-related parameters of the initial refresh area meet the preset conditions is used as the reference data frame.

[0194] In one exemplary embodiment, the third determining module 903 is further configured to:

[0195] The end data frame is determined based on the reference data frame and the number of data frames that refresh the display screen once.

[0196] In one exemplary embodiment, the fourth determining module 904 is further configured to:

[0197] The total number of frames included in the unlocking process is determined based on the initial data frame and the ending data frame.

[0198] The unlocking time is determined based on the total number of frames included in the unlocking process and the frame rate at which the data frames are acquired.

[0199] In an exemplary embodiment, the first determining module 901 is further configured to:

[0200] Based on the rectangle detection algorithm, the vertex coordinates of the display screen in the first data frame are determined;

[0201] Based on the vertex coordinates, the recognition area of ​​the display screen in each data frame is determined.

[0202] In an exemplary embodiment, the first determining module 901 is further configured to:

[0203] All data frames in the unlocking process are converted from RGB images to HSV images.

[0204] In one exemplary embodiment, the second determining module 902 is further configured to:

[0205] Determine the sum of the V values ​​of all pixels in the fingerprint unlocking area;

[0206] Obtain the initial frame data frame when the sum of the V values ​​of all pixels in the fingerprint unlocking area is greater than or equal to the second preset threshold;

[0207] The initial data frame is used as the initial data frame.

[0208] In one exemplary embodiment, the displayed image is a preset displayed image;

[0209] The third determining module 903 is further configured to:

[0210] Obtain the H values ​​of all pixels in the initial refresh area of ​​the displayed image after the display screen is unlocked;

[0211] The first frame of data when the sum of the H values ​​of all pixels in the initial refresh area equals a preset H value is used as the reference data frame.

[0212] In an exemplary embodiment, the first determining module 901 is further configured to:

[0213] Based on the Hough circle detection algorithm, the center and radius of the circle in the recognition area of ​​the display screen are detected;

[0214] Using the center of the circle as the center of the fingerprint unlocking area, a circle with a radius within a preset range is defined as the fingerprint unlocking area.

[0215] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0216] Figure 10 This is a block diagram of a fingerprint unlocking time determination device 1000 according to an exemplary embodiment.

[0217] Reference Figure 10 The device 1000 may include one or more of the following components: a processing component 1002, a memory 1004, a power supply component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.

[0218] Processing component 1002 typically controls the overall operation of device 1000, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1002 may include one or more processors 1020 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1002 may include one or more modules to facilitate interaction between processing component 1002 and other components. For example, processing component 1002 may include a multimedia module to facilitate interaction between multimedia component 1008 and processing component 1002.

[0219] Memory 1004 is configured to store various types of data to support the operation of device 1000. Examples of such data include instructions for any application or method operating on device 1000, contact data, phonebook data, messages, pictures, videos, etc. Memory 1004 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0220] Power supply component 1006 provides power to various components of device 1000. Power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 1000.

[0221] The multimedia component 1008 includes a screen that provides an output interface between the device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 1008 includes a front-facing camera and / or a rear-facing camera. When the device 1000 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0222] Audio component 1010 is configured to output and / or input audio signals. For example, audio component 1010 includes a microphone (MIC) configured to receive external audio signals when device 1000 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1004 or transmitted via communication component 1016. In some embodiments, audio component 1010 also includes a speaker for outputting audio signals.

[0223] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0224] Sensor assembly 1014 includes one or more sensors for providing state assessments of various aspects of device 1000. For example, sensor assembly 1014 may detect the on / off state of device 1000, the relative positioning of components such as the display and keypad of device 1000, changes in the position of device 1000 or a component of device 1000, the presence or absence of user contact with device 1000, the orientation or acceleration / deceleration of device 1000, and temperature changes of device 1000. Sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1014 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1014 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0225] Communication component 1016 is configured to facilitate wired or wireless communication between device 1000 and other devices. Device 1000 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 1016 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1016 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0226] In an exemplary embodiment, the apparatus 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0227] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1004 including instructions, which can be executed by a processor 1020 of the device 1000 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0228] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a device, enables the device to perform a method for determining a fingerprint unlock time, the method comprising any of the methods described above.

[0229] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0230] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for determining fingerprint unlocking time, applied to a terminal, characterized in that, The determination method includes: The identification area of ​​the terminal's display screen is determined, and the identification area includes a fingerprint unlocking area; Determine the initial data frame corresponding to the unlocking process of the fingerprint unlocking area, wherein the initial data frame in which the brightness value of the fingerprint unlocking area is greater than or equal to a first preset threshold during the unlocking process is taken as the initial data frame; The fingerprint unlocking area corresponds to the end data frame of the unlocking process, wherein the end data frame is determined based on the difference features of the images displayed in the recognition area of ​​the terminal's display screen before and after unlocking; The unlocking time is determined based on the initial data frame and the end data frame.

2. The method for determining fingerprint unlocking time according to claim 1, characterized in that, The step of determining the end data frame of the unlocking process corresponding to the fingerprint unlocking area includes: According to the refresh method of the display screen, an initial refresh area is determined in the identification area, and the size of the initial refresh area is 1 / n of the size of the identification area; where n is the number of data frames that refresh the display screen once. The first data frame that makes the preset parameters of the initial refresh area meet the preset conditions will be used as the reference data frame. The end data frame is determined based on the reference data frame.

3. The method for determining fingerprint unlocking time according to claim 2, characterized in that, Determining the initial refresh area in the identification area according to the refresh mode of the display screen includes: When the refresh mode of the display screen is to refresh from the first position to the second position along a preset direction, the starting position of the first position is taken as the starting position of the initial refresh area to determine the initial refresh area.

4. The method for determining fingerprint unlocking time according to claim 3, characterized in that, The step of using the data frame that makes the preset parameters of the initial refresh area meet the preset conditions as the reference data frame includes: Based on the displayed image after the display screen is unlocked, determine the color-related parameters of the initial refresh area; The first data frame when the color-related parameters of the initial refresh area meet the preset conditions is used as the reference data frame.

5. The method for determining fingerprint unlocking time according to claim 3, characterized in that, Determining the end data frame based on the reference data frame includes: The end data frame is determined based on the reference data frame and the number of data frames that refresh the display screen once.

6. The method for determining fingerprint unlocking time according to claim 1, characterized in that, Determining the unlock time based on the initial data frame and the end data frame includes: The total number of frames included in the unlocking process is determined based on the initial data frame and the ending data frame. The unlocking time is determined based on the total number of frames included in the unlocking process and the frame rate at which the data frames are acquired.

7. The method for determining fingerprint unlocking time according to claim 1, characterized in that, Determining the recognition area of ​​the terminal's display screen includes: Based on the rectangle detection algorithm, the vertex coordinates of the display screen in the first data frame are determined; Based on the vertex coordinates, the recognition area of ​​the display screen in each data frame is determined.

8. The method for determining fingerprint unlocking time according to claim 1, characterized in that, The method further includes: All data frames in the unlocking process are converted from RGB images to HSV images.

9. The method for determining fingerprint unlocking time according to claim 8, characterized in that, The step of using an initial data frame whose brightness value in the fingerprint unlocking area is greater than or equal to a first preset threshold during the unlocking process as the initial data frame includes: Determine the sum of the V values ​​of all pixels in the fingerprint unlocking area; Obtain the initial frame data frame when the sum of the V values ​​of all pixels in the fingerprint unlocking area is greater than or equal to the second preset threshold; The initial data frame is used as the initial data frame.

10. The method for determining fingerprint unlocking time according to claim 4, characterized in that, The displayed image is a preset display image; The step of using the first frame of data when the color-related parameters of the initial refresh area satisfy the preset condition as the reference data frame includes: Obtain the H values ​​of all pixels in the initial refresh area of ​​the displayed image after the display screen is unlocked; The first frame of data when the sum of the H values ​​of all pixels in the initial refresh area equals a preset H value is used as the reference data frame.

11. The method for determining fingerprint unlocking time according to any one of claims 1-10, characterized in that, The method further includes: Based on the Hough circle detection algorithm, the center and radius of the circle in the recognition area of ​​the display screen are detected; Using the center of the circle as the center of the fingerprint unlocking area, a circle with a radius within a preset range is defined as the fingerprint unlocking area.

12. A device for determining fingerprint unlocking time, applied to a terminal, characterized in that, The determining device includes: The first determining module is configured to determine the recognition area of ​​the terminal's display screen, the recognition area including a fingerprint unlocking area; The second determining module is configured to determine the initial data frame of the unlocking process corresponding to the fingerprint unlocking area, wherein the initial data frame in which the brightness value of the fingerprint unlocking area is greater than or equal to a first preset threshold during the unlocking process is taken as the initial data frame. The third determining module is configured to determine the end data frame of the unlocking process corresponding to the fingerprint unlocking area, wherein the end data frame is determined based on the difference features of the images displayed in the recognition area of ​​the terminal's display screen before and after unlocking; The fourth determining module is configured to determine the unlocking time based on the initial data frame and the end data frame.

13. A terminal, characterized in that, include: processor; Memory used to store processor-executable instructions; Among them, the The processor is configured to perform the method as described in any one of claims 1-11.

14. A non-transitory computer-readable storage medium, wherein when instructions in the storage medium are executed by a processor of a device, characterized in that, This enables the device to perform the method as described in any one of claims 1-11.

Images