Detection method, device, electronic device and storage medium

Abstract

This disclosure relates to a detection method, apparatus, electronic device, and storage medium. The detection method includes: receiving an image sequence sent by a terminal in response to an action sequence, the image sequence including multiple frames of images; sequentially acquiring an action content from the action sequence as the current action content, and performing the following operations: determining a starting image corresponding to the current action content; sequentially determining the action scores of each image in the image sequence following the starting image and the current action content; determining a matching result corresponding to the current action content based on the action score of any image; determining a matching result between the image sequence and the action sequence based on the matching results corresponding to all action content in the action sequence; and generating a detection result based on the matching result between the image sequence and the action sequence. This disclosure can improve the security of the user's detection environment and the accuracy of the detection results.

Description

Technical Field

[0001] This disclosure relates to the field of testing, and more particularly to a testing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In scenarios requiring human verification, such as online finance and account logins, operators prefer that users who pass the verification are genuine account owners, rather than those using scripts or impersonators. If a user is using a script or impersonator in a human verification scenario, the verification is highly likely to be malicious, meaning the verification environment is insecure and could easily lead to financial losses for the user. Therefore, improving the security of the verification environment is a pressing issue that needs to be addressed. Summary of the Invention

[0003] This disclosure proposes a detection technology solution.

[0004] According to one aspect of this disclosure, a detection method is provided, applied to a server. The detection method includes: receiving an image sequence sent by a terminal in response to an action sequence, the image sequence including multiple frames of images; sequentially acquiring an action content from the action sequence as the current action content, and performing the following operations: determining a starting image corresponding to the current action content; sequentially determining the action scores of each image in the image sequence following the starting image and the current action content; determining a matching result corresponding to the current action content based on the action score of any image; determining a matching result between the image sequence and the action sequence based on the matching results corresponding to all action content in the action sequence; and generating a detection result based on the matching result between the image sequence and the action sequence.

[0005] In one possible implementation, determining the starting image corresponding to the current action content includes: if the current action content is determined to be the first action content in the action sequence, the starting image is the starting image of the image sequence; if the current action content is not determined to be the first action content in the action sequence, the starting image is the next frame image of the image that successfully matches the previous action content.

[0006] In one possible implementation, determining the matching result between the image sequence and the action sequence includes: if no matching result of the action sequence is obtained within a first preset time period starting from the start of determining the matching result of the action sequence, and / or if no matching result of the action content is obtained within a second preset time period starting from the start of determining the matching result of any action content in the action sequence, determining the matching result between the image sequence and the action sequence as a matching failure.

[0007] In one possible implementation, generating a detection result based on the matching result of the image sequence and the action sequence includes: if the matching result of the image sequence and the action sequence is determined to be a successful match, filtering out a first image from the image sequence; generating a liveness detection result based on the first image; and determining the detection result based on the liveness detection result, wherein if the liveness detection result is determined to be a live object, the detection result is considered a successful detection.

[0008] In one possible implementation, the step of filtering out the first image in the image sequence includes: filtering out a preset number of first images in the image sequence whose action scores are greater than or equal to a second score threshold.

[0009] In one possible implementation, generating a liveness detection result based on the first image includes: generating a liveness detection sub-result corresponding to the first image; using the first image with the highest action score as the second image; and determining that the liveness detection result is live when the liveness detection sub-result corresponding to the second image is determined to be live and the ratio of the number of first images with liveness detection sub-results to the total number of all first images is greater than or equal to a preset ratio.

[0010] In one possible implementation, the receiving terminal responding to the image sequence sent by the action sequence includes: decrypting the image sequence sent by the terminal in response to the action sequence to obtain a decrypted image sequence; the step of sequentially determining the action score of each image after the starting image in the image sequence and the action score of the current action content includes: sequentially determining the action score of each image after the starting image in the decrypted image sequence and the action score of the current action content.

[0011] In one possible implementation, determining the matching result between the image sequence and the action sequence further includes: generating at least one of facial region coordinates and facial number corresponding to the images in the image sequence; and determining the matching result based on the facial region coordinates, the facial number, the image sequence, and the action sequence.

[0012] In one possible implementation, determining the matching result based on at least one of the facial region coordinates and the facial number, as well as the image sequence and the action sequence, includes: determining the matching result between the facial region indicated by the facial region coordinates and the action sequence in each image of the image sequence, and using this as the matching result between the image sequence and the action sequence.

[0013] In one possible implementation, determining the matching result based on at least one of the facial region coordinates and the facial ID, as well as the image sequence and the action sequence, further includes: if the number of images corresponding to the least frequent facial ID in the image sequence is greater than a first threshold, determining the matching result as a matching failure.

[0014] In one possible implementation, the detection method further includes at least one of the following: if the detection result is determined to be a detection failure, sending a first instruction to the terminal, the first instruction controlling the terminal to enter a page for resending the detection request; if the number of times the first instruction is sent to the terminal within a third preset time period reaches a second threshold, in response to a detection request sent by the terminal through the page, sending a second instruction to the terminal, the second instruction notifying the terminal that the server rejects the detection request; and if the time from sending the first instruction to receiving a new image sequence is determined to be greater than a fourth preset time, sending the second instruction to the terminal.

[0015] According to one aspect of this disclosure, a detection device is provided, applied to a server, the detection device comprising: an image sequence receiving module, configured to receive an image sequence sent by a terminal in response to an action sequence, the image sequence including multiple frames of images; an action content processing module, configured to sequentially acquire an action content from the action sequence as the current action content, and perform the following operations: determining a starting image corresponding to the current action content; sequentially determining the action scores of each image in the image sequence following the starting image and the current action content; determining a matching result corresponding to the current action content based on the action score of any image; determining a matching result between the image sequence and the action sequence based on the matching results corresponding to all action content in the action sequence; and a detection result generation module, configured to generate a detection result based on the matching result between the image sequence and the action sequence.

[0016] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the detection method described in any one of the preceding claims.

[0017] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the detection method described in any one of the preceding claims.

[0018] This disclosure provides a detection method in which a server receives an image sequence sent by a terminal in response to an action sequence. The image sequence includes multiple frames. Then, a single action content from the action sequence is sequentially acquired as the current action content, and the following operations are performed: determining the starting image corresponding to the current action content; sequentially determining the action scores of each image in the image sequence following the starting image and the current action content; determining the matching result corresponding to the current action content based on the action score of any image; determining the matching result between the image sequence and the action sequence based on the matching results of all action content in the action sequence; and finally generating a detection result based on the matching result between the image sequence and the action sequence. Since the detection result is generated on the server, the possibility of malicious programs altering the detection result is reduced. Combined with the aforementioned action scores, the accuracy of the matching result can be further improved, thereby achieving accurate detection of the security of the verification environment. Furthermore, since the server has higher computing power than the terminal, the server can reduce the time required to generate the detection result or use a more computationally complex but more accurate detection model.

[0019] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0021] Figure 1 A flowchart of a detection method provided according to an embodiment of this disclosure is shown.

[0022] Figure 2 A flowchart of a detection method provided according to an embodiment of this disclosure is shown.

[0023] Figure 3 A block diagram of a detection apparatus provided according to an embodiment of the present disclosure is shown.

[0024] Figure 4 A block diagram of an electronic device provided according to an embodiment of the present disclosure is shown. Detailed Implementation

[0025] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0026] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0027] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0028] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0029] In related technologies, detection technology is usually built into the terminal's application. Therefore, the detection process is generally as follows: the terminal receives the user's detection request, identifies the user's image data through the built-in detection technology, generates a detection result, and then sends the detection result to the server. The server determines whether to provide further services to the terminal based on the detection result.

[0030] However, this setup can easily lead to the following problems: 1. The detection results are generated by the terminal and then transmitted to the server, making them susceptible to tampering by malicious programs. For example, the terminal might detect a failed match, but a malicious program could modify it to show a successful match and then send it to the server. The server would then believe it can provide further services to the terminal, meaning it considers the terminal's detection environment secure. However, in reality, the terminal's detection environment is not secure, and malicious programs can easily cause financial losses to users. 2. The detection technology is integrated into the application, and the computing power of the detection technology used is limited. With limited computing power, to reduce user waiting time, the application is unlikely to use more computationally complex detection technologies, thus limiting its detection accuracy.

[0031] In view of this, this disclosure provides a detection method in which a server receives an image sequence sent by a terminal in response to an action sequence, the image sequence including multiple frames of images, and then sequentially acquires one action content from the action sequence as the current action content, and performs the following operations: determining the starting image corresponding to the current action content; sequentially determining the action scores of each image in the image sequence after the starting image and the current action content; determining the matching result corresponding to the current action content based on the action score of any image; determining the matching result between the image sequence and the action sequence based on the matching results corresponding to all action content in the action sequence; and finally generating a detection result based on the matching result between the image sequence and the action sequence. Since the above detection result is generated on the server, the possibility of malicious programs altering the detection result is reduced. Combined with the above action scores, the accuracy of the matching result can be further improved, thereby achieving accurate detection of the security of the verification environment. Furthermore, since servers have higher computing power than terminals, servers can reduce the time required to generate detection results or use more complex but more accurate detection models. Terminals can display pages such as recorded image sequences, retries, and detection results in the form of H5 web pages, making the terminal-side program lightweight and reducing the requirements for terminal computing power.

[0032] For example, the above detection method is executed by a server, such as a physical server, virtual host, virtual private server (VPS), cloud server, etc. The server interacts with a terminal, which can be a mobile device, user terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, the above detection method can also be implemented by a processor calling computer-readable instructions stored in memory.

[0033] See Figure 1 As shown, Figure 1 A flowchart illustrating a detection method according to an embodiment of this disclosure is shown, such as... Figure 1 As shown, the above detection method includes the following steps:

[0034] In step S100, the receiving terminal responds to the image sequence sent by the action sequence, the image sequence including multiple frames of images. For example, before performing this step, the server may, in response to a detection request sent by the terminal, randomly select action content to generate an action sequence. In one example, the action sequence includes multiple action contents, each indicating a facial action that the user needs to perform. The server can obtain the action contents through a preset action content library storing multiple action contents. In one example, the server can randomly select a fixed number of action contents from the preset action content library and randomly sort each action content (that is, the server can randomly send different numbers and different orders of action sequences to the terminal each time it responds to a detection request sent by the terminal). For example, if the server is set to select 3 action contents, it will select 3 action contents from blinking, shaking head, nodding, opening mouth, tilting head, and smiling, and after shuffling the order of the action contents, obtain a random action sequence. In this embodiment, the action sequence is randomly generated. This means that the action sequence obtained by the terminal after each detection request is highly likely to be different, thus reducing the possibility of malicious software pre-recording image sequences and improving the security of the detection environment. Continuing the example, the server can also select a random number of action contents and randomly sort each action content to further improve the security of the detection environment. For example, if the server is set to select 2 to 5 random action contents, it can randomly select several action contents from blinking, shaking head, nodding, opening mouth, tilting head, and smiling, and after shuffling the order of the action contents, obtain a random action sequence. The server can then send the action sequence to the terminal. Exemplarily, after receiving the action sequence, the terminal prompts the user with the action sequence, for example, through voice or text prompts. The user then follows the action sequence to start recording an image sequence through the terminal. After recording, the terminal sends the recorded image sequence to the server. Exemplarily, the terminal can also limit and prompt the user with the maximum recording time of the image sequence to save server computing power. In one example, the terminal can interact with the user via a webpage to achieve a lightweight detection method. The image sequence can be video or continuously captured images.

[0035] In one possible implementation, the terminal may send an encrypted image sequence to the server to improve the security of image sequence transmission. In this case, step S100 may include: decrypting the image sequence sent by the terminal in response to the action sequence to obtain a decrypted image sequence. Then, various steps may be performed based on the decrypted image sequence. This disclosure embodiment, by encrypting the image sequence, can reduce the risk of the image sequence being altered by other malicious programs, thereby improving the security of the image sequence during transmission. Exemplarily, the image sequence may be encrypted frame by frame to further increase the security of the image sequence during transmission.

[0036] Continue reading Figure 1 In step S200, one action content from the action sequence is sequentially obtained as the current action content, and the following operations are performed:

[0037] In step S210, the starting image corresponding to the current action content is determined, and the action scores of each image after the starting image in the image sequence and the current action content are determined sequentially. The action score can be positively correlated with the standard degree of the action content in the image relative to the current action content, and can be obtained through a machine learning model. For example, if the machine learning model is a binary classification model (i.e., each input image is classified as 'is the action content' or 'is not the action content'), then in the process of classifying the action content in an image, the machine learning model first generates an action score for the input image, and then when the action score of the input image is greater than or equal to a score threshold, the input image is classified as 'is the action content'. In other words, the action score mentioned in this embodiment can be equal to the action score used by the machine learning model in the classification process. For example, when the current action content is determined to be the first action content in the action sequence, the starting image is the starting image of the image sequence, for example, the first frame image is used as the starting image of the image sequence, or a certain frame image in the image sequence is pre-specified as the starting image. If the current action content is not the first action content in the action sequence, the starting image is the next frame image after the image that successfully matches the previous action content. This embodiment of the disclosure reduces the computational load of action content matching by setting a starting image, and the starting image can also serve as a marker for the order of action content in the image sequence. For example, if the image sequence consists of 20 frames, and the action sequence includes blinking, opening the mouth, and tilting the head, the server will use the first frame image as the starting image for the blinking action and generate an action score. If the blinking action is successfully matched in the 6th frame, then the 7th frame will be used as the starting image for the opening the mouth action. If the opening the mouth action is successfully matched in the 12th frame, then the 13th frame will be used as the starting image for the tilting the head action. If the starting image for the tilting the head action is successfully matched in the 15th frame, then it is unnecessary to detect the images from the 16th to the 20th frames, thus saving server computing power. Then, the action content corresponding to the successfully matched 6th, 12th, and 15th frames will be used as the order of action content in the image sequence. In one possible implementation, if the image sequence is encrypted by the terminal device, step S210 may include: sequentially determining the action scores of each image after the starting image in the decrypted image sequence and the current action content. This embodiment of the disclosure, by encrypting the image sequence, can reduce the risk of the image sequence being altered by other malicious programs, thereby improving the security of the image sequence during transmission.

[0038] In step S220, the matching result corresponding to the current action content is determined based on the action score of any image. For example, when the action score of any image is greater than a first score threshold, the matching result corresponding to the current action content is determined to be a successful match. Server administrators can set the aforementioned first score threshold according to actual conditions. For example, the higher the aforementioned first score threshold, the more standardized the corresponding action content in the image needs to be, and the more accurate the final matching result will be. This embodiment of the disclosure does not limit the specific value of the first score threshold. For example, if no successful matching result is obtained within a first preset time period starting from the determination of the matching result of the action sequence, and / or if no successful matching result is obtained within a second preset time period starting from the determination of the matching result of any action content in the action sequence, the matching result of the image sequence and the action sequence is determined to be a failed match. For example, if the first preset time is 20 seconds, and the server does not complete the matching of each action content in the action sequence within 20 seconds, the server determines that the image sequence matching has failed. If the second preset time is 5 seconds, and the server fails to match any action content in the action sequence within 5 seconds (i.e., if a certain action content in the image sequence fails to match after 5 seconds), then the server determines that the image sequence matching has failed. By setting the above conditions, the security of the user verification stage can be further increased, while improving verification efficiency. The specific values ​​of the first and second preset times are not limited in this embodiment.

[0039] In step S230, the matching result between the image sequence and the action sequence is determined based on the matching results corresponding to all action content in the action sequence. For example, when all matching results corresponding to action content in the action sequence are successful, the matching result between the image sequence and the action sequence is determined to be successful. For example, the matching result between the image sequence and the action sequence can be determined by a machine learning model. Each action content in the action sequence corresponds to a machine learning model. The server sequentially calls the machine learning models corresponding to the action content in the action sequence to achieve image sequence matching detection. This embodiment does not limit the training method of the machine learning models; each machine learning model only needs to be able to detect the corresponding action content. For example, the input of the machine learning model can be an image, and the output can be the matching result of the action content corresponding to that machine learning model. For example, the machine learning model can determine whether the action content is successfully matched by extracting the positional feature relationships between facial key points in the image (e.g., by using algorithms such as Active Shape Model, Active Appearance Models, cascaded pose regression algorithms, and temporal action detection algorithms). For example, in this embodiment of the disclosure, the machine learning model is integrated into a server with high computing power, rather than in the terminal. That is, the detection method of this embodiment can use a more computationally complex but more accurate machine learning model. For example, a machine learning model with continuous image matching logic from related technologies can be used, thereby making the matching results more accurate. For instance, for an image that successfully matches the 'open mouth' action, the vertical distance of the mouth key points in the previous image should be smaller than the vertical distance of the mouth key points in the current image (i.e., the user's face has gone from a 'closed mouth state' to an 'open mouth state' in multiple consecutive frames). The above-mentioned machine learning model can be referred to related technologies, and will not be elaborated here. In other words, in this embodiment of the disclosure, the server can perform action detection and liveness detection frame by frame to introduce the correlation information between consecutive frames to increase the accuracy of the detection results. For example, various time limits can also be added to the detection process (which will be detailed later) to further increase the security of the user's environment.

[0040] In one possible implementation, step S200 may include determining that the matching result is successful when it is determined that the action content in the action sequence matches the action content detected in the image sequence one by one and in the same order. For example, if the action sequence includes blinking, shaking head, and opening mouth in sequence, then the order of the action content in the image sequence should follow the order of blinking, shaking head, and opening mouth. If the order of the action content in the image sequence is blinking, opening mouth, and shaking head, then the matching result is determined to be a failure; if the order of the action content in the image sequence is blinking and shaking head, then the matching result is also determined to be a failure. This embodiment of the present disclosure can accurately determine whether the user's detection environment is safe by detecting the number and order of action content.

[0041] In one possible implementation, if for the purpose of saving computing power and improving user security, the above determination of the matching result between the image sequence and the action sequence may include: generating at least one of the facial region coordinates and facial number corresponding to the images in the image sequence; and determining the matching result based on the facial region coordinates, the facial number, the image sequence, and the action sequence.

[0042] Exemplarily, facial region coordinates can be obtained through facial region extraction models in related technologies, and this disclosure does not limit this embodiment. The aforementioned facial region coordinates are used to indicate the user's facial region in each image of the image sequence. In one example, determining the matching result can be determining the matching result between the facial region indicated by the facial region coordinates and the action sequence in each image of the image sequence, and using this as the matching result between the image sequence and the action sequence. For example, facial region can be used instead of "image" in the above, and steps S200 and S300 can be executed to obtain the detection result. By setting facial region coordinates, local matching of images can be achieved, thereby reducing the computational power consumption of the server.

[0043] The aforementioned facial IDs are used to distinguish users with different facial features. These IDs can be obtained through the facial region extraction model described above, ensuring that the facial images in the image sequence largely originate from the same user. In other words, if the image sequence contains images of both user A and user B, the facial IDs corresponding to user A's facial region image and user B's facial region image will be different. In one example, if the number of images corresponding to the minimum facial ID in the image sequence exceeds a first threshold, the matching result is determined to be a match failure. For example, if the image sequence contains 15 frames of user A's images and 20 frames of user B's images, and the first threshold is 10 frames, the server determines the matching result to be a match failure (i.e., 15 frames is greater than 10 frames), to avoid situations where user A and user B are simultaneously verifying. With the above settings, the server can allow for a certain degree of unexpected situations (e.g., the terminal's camera capturing a face behind the user) during image sequence acquisition, while ensuring the security of the user verification environment.

[0044] Continue reading Figure 1 In step S300, a detection result is generated based on the matching result between the image sequence and the action sequence.

[0045] In one possible implementation, the final detection result can be generated based on the matching result and the liveness detection result.

[0046] See Figure 2 As shown, Figure 2 A flowchart illustrating a detection method according to an embodiment of this disclosure is shown. Figure 2 As shown, in one possible implementation, step S300 may include:

[0047] In step S310, if the matching result between the image sequence and the action sequence is determined to be a successful match, a first image is selected from the image sequence. In one example, this step may be: selecting a preset number of first images from the image sequence whose action scores are greater than or equal to a second score threshold. This embodiment of the present disclosure can use images with higher action scores from the selected image sequence as images for subsequent liveness detection, thereby saving server computing power. Furthermore, images with higher action scores are usually representative, so the selection of images has a relatively small impact on the accuracy of the liveness detection results.

[0048] For example, the second scoring threshold can be less than or equal to the first scoring threshold, and the first and second scoring thresholds can be different for different action content. For instance, if the image sequence includes: image A (score of 20), image B (score of 40), image C (score of 60), image D (score of 80), image E (score of 30), image F (score of 45), image G (score of 70), and image H (score of 80), and images A to D belong to the same action content (the first scoring threshold for this action content is 65), and images E to H belong to the same action content (the first scoring threshold for this action content is 75), and the second scoring threshold is 50 for all of them, and the preset number is 3, then according to the principle of selecting up to the preset number, images C, D, and G are used as the first images, i.e., image H is discarded. If the calculation time is not considered, all images with action scores greater than the second scoring threshold can be obtained, and the images with the lowest scores can be discarded, i.e., the preset number of images with the highest action scores are retained to improve the accuracy of liveness detection, such as discarding image C. If the second scoring threshold for the first action content is 30, the second scoring threshold for the second action content is 40, and the preset quantity is 6, then images B, C, D, F, G, and H are selected as the first images. For example, the preset quantity can represent the total number of first images with an action score greater than or equal to the second scoring threshold, or it can represent the number of images with an action score greater than or equal to the second scoring threshold for each action content. Continuing the example, if the preset quantity for each action content is 2, then images B, C, D, F, G, and H are selected as images B, C, F, and G. Without considering the calculation time, the images with the highest scores, such as images C, D, G, and H, can also be selected.

[0049] In step S320, a liveness detection result is generated based on the first image. The filtered first image not only has higher image quality (i.e., is more likely to contain live objects), but also has fewer images than the image sequence, which can effectively reduce the computation time for liveness detection.

[0050] In one possible implementation, step S320 may include: generating a liveness detection sub-result corresponding to the first image based on the first image. The first image with the highest action score is used as the second image. If the liveness detection sub-result corresponding to the second image is determined to be live, and the ratio of the number of first images with liveness detection sub-results to the total number of first images is greater than or equal to a preset ratio, the liveness detection result is determined to be live. In this embodiment, the above detection rule is defined: if the second image is live, and the proportion of images with liveness detection results is greater than or equal to a preset ratio, the server determines that the liveness detection result of the image sequence is live. In actual shooting scenarios, when a user takes pictures of an image sequence through a terminal, there is a certain probability of interference from external factors, such as someone else's face being unintentionally captured by the camera, or the terminal being dropped. Therefore, in the above situations, the image sequence may contain non-live images. This embodiment, through the above detection rule, allows the image sequence to contain a certain number of non-live images. However, if the number of non-live images exceeds a preset ratio, there is a high probability of malicious detection. For example, someone could maliciously create a mask of the account holder. If this mask fits a face, others could easily perform various actions of the account holder. To address this, this embodiment of the disclosure reduces the probability of such detections succeeding by setting a liveness detection method, thereby improving the security of user verification. The preset ratio can be set according to actual conditions, and this embodiment of the disclosure does not impose any limitations. For example, the higher the preset ratio, the higher the proportion of non-live images, and the higher the probability of a liveness detection result indicating a live person.

[0051] For example, the above-mentioned liveness detection sub-result can be generated by a machine learning model in related technologies. The machine learning model can generate liveness detection sub-result based on an image or a face region image in an image. For example, the machine learning model can extract features such as color texture, non-rigid motion deformation, face material, and image distortion rate of live and non-live objects to generate liveness detection sub-results. The embodiments disclosed herein will not be elaborated further.

[0052] In step S330, based on the liveness detection result, the detection result is determined, wherein if the liveness detection result indicates a live subject, the detection result is considered successful. That is, when the matching result between the image sequence and the action sequence is successful, and the liveness detection result indicates a live subject, the detection result is considered successful. The combination of action matching and liveness detection further improves the accuracy of the verification. Furthermore, this embodiment avoids the insecurity of silent liveness detection used in related technologies by using a combination of action and liveness detection.

[0053] In one possible implementation, the detection method further includes sending the detection result to the terminal. For example, when the detection result is "passed," the server allows the terminal to perform further operations (e.g., entering a payment password, changing an account password, granting specific permissions, etc.). After receiving the detection result, the terminal prompts the user that the detection has passed and further operations can be performed. For example, after the detection result is generated, the service provider can obtain the detection result through the server's interface and then determine whether to provide the corresponding service to the terminal. That is, the service provider can use its own server and the server in this embodiment of the disclosure to provide various services.

[0054] For example, when the detection result is determined to be a failure, a first instruction is sent to the terminal. This first instruction controls the terminal to enter a page for resending the detection request. Upon receiving the first instruction, the terminal can enter the page for resending the detection request and prompt the user that the detection failed and whether a resend request is needed. This prompt can last for a certain period. If the user resends the detection request through the terminal, the detection method of this embodiment is re-executed from step S100 or its preceding steps. During each retry, the server can generate different action sequences to reduce the possibility of malicious software pre-generating image sequences that pass detection, thereby improving the security of the user's environment.

[0055] In one example, if the number of times the first instruction is sent to the terminal within a third preset time period reaches a second threshold, in response to a detection request sent by the terminal through the page, a second instruction is sent to the terminal. This second instruction notifies the terminal that the server rejects the detection request. Upon receiving the second instruction, the terminal indicates to the user that the detection failed, and the server refuses the terminal from initiating a retry through the aforementioned retry page. Correspondingly, if the number of times the terminal sends the first instruction does not reach the second threshold, but the third preset time has been reached, the terminal may no longer display the page for resending the detection request; that is, the user can no longer send a detection request through this page.

[0056] The aforementioned third preset time can be calculated from the moment the terminal first makes a detection request in this overall detection process. For example, if the aforementioned third preset time is 10 minutes, the timer starts when the user opens a webpage on the terminal and sends the first detection request. When the timer exceeds 10 minutes, the user cannot submit a detection request again on the page used for retrying. If the aforementioned second threshold is 5 times, and the server issues the first instruction 5 times within 10 minutes, it means that the user has retried 5 times and all tests have failed. The server will reject subsequent detection requests sent by the terminal on that page. For example, the detection request sent by the terminal when initiating a retry can carry a request identifier. For example, the request identifier can be an incrementing request identifier, which is incremented by 1 each time a retry is initiated. The server can determine that the detection request comes from the aforementioned page, belongs to the detection request in the retry process, and determine the number of retryes by the terminal (i.e., the number of times the server sends the first instruction).

[0057] If configured as described above, the cost for an attacker to crack the detection method provided in this disclosure embodiment can be increased. For example, after 10 minutes of the first submission of a detection request or after 5 retries, the attacker will no longer be able to submit a detection request again through the same webpage (such as the page mentioned above used to resend the detection request). If the attacker wants to continue experiencing the above detection method in order to attempt to crack it, they need to open a new webpage. If the attacker opens the webpage too many times, the IP address corresponding to the terminal will have multiple records of accessing the webpage. The owner or all entities of the terminal can then use the security detection methods in related technologies to promptly detect that the terminal is performing malicious operations, thereby increasing the probability of the terminal being detected when performing malicious operations, which in turn increases the cost for the attacker to crack the method.

[0058] In one example, if the time from sending the first instruction to receiving the new image sequence is determined to be greater than a fourth preset time, a second instruction is sent to the terminal. For example, the fourth preset time could be 1 minute, meaning the user needs to complete recording the image sequence within 1 minute. This shortens the available time for an attacker to maliciously use video editing software to generate a synthetic image sequence, thereby reducing the likelihood of an attacker using a synthetic image sequence and further increasing the security of the user's detection environment.

[0059] This disclosure reduces the time attackers spend preparing synthetic image sequences by establishing retry rules for detection requests, thereby increasing the security of the user's detection environment.

[0060] This disclosure does not limit the specific values ​​of the third preset time, the fourth preset time, and the second threshold mentioned above. Service providers can determine the specific data according to actual needs.

[0061] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0062] In addition, this disclosure also provides a detection device, electronic device, computer-readable storage medium, and program, all of which can be used to implement any of the detection methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding records in the method section and will not be repeated here.

[0063] See Figure 3 As shown, Figure 3 A block diagram of a detection apparatus according to an embodiment of the present disclosure is shown. Figure 3 As shown, in one possible implementation, this disclosure also provides a detection device 100 applied to a server. The detection device includes: an image sequence receiving module 110, used to receive an image sequence sent by a terminal in response to the action sequence, the image sequence including multiple frames of images; an action content processing module 120, used to sequentially acquire one action content from the action sequence as the current action content, and perform the following operations: determine the starting image corresponding to the current action content; sequentially determine the action scores of each image after the starting image in the image sequence and the current action content; determine the matching result corresponding to the current action content based on the action score of any image; determine the matching result between the image sequence and the action sequence based on the matching results corresponding to all action content in the action sequence; and a detection result generation module 130, used to generate a detection result based on the matching result between the image sequence and the action sequence.

[0064] In one possible implementation, determining the starting image corresponding to the current action content includes: if the current action content is determined to be the first action content in the action sequence, the starting image is the starting image of the image sequence; if the current action content is not determined to be the first action content in the action sequence, the starting image is the next frame image of the image that successfully matches the previous action content.

[0065] In one possible implementation, determining the matching result between the image sequence and the action sequence includes: if no matching result of the action sequence is obtained within a first preset time period starting from the start of determining the matching result of the action sequence, and / or if no matching result of the action content is obtained within a second preset time period starting from the start of determining the matching result of any action content in the action sequence, determining the matching result between the image sequence and the action sequence as a matching failure.

[0066] In one possible implementation, generating a detection result based on the matching result of the image sequence and the action sequence includes: if the matching result of the image sequence and the action sequence is determined to be a successful match, filtering out a first image from the image sequence; generating a liveness detection result based on the first image; and determining the detection result based on the liveness detection result, wherein if the liveness detection result is determined to be a live object, the detection result is considered a successful detection.

[0067] In one possible implementation, the step of filtering out the first image in the image sequence includes: filtering out a preset number of first images in the image sequence whose action scores are greater than or equal to a second score threshold.

[0068] In one possible implementation, generating a liveness detection result based on the first image includes: generating a liveness detection sub-result corresponding to the first image; using the first image with the highest action score as the second image; and determining that the liveness detection result is live when the liveness detection sub-result corresponding to the second image is determined to be live and the ratio of the number of first images with liveness detection sub-results to the total number of all first images is greater than or equal to a preset ratio.

[0069] In one possible implementation, the receiving terminal responding to the image sequence sent by the action sequence includes: decrypting the image sequence sent by the terminal in response to the action sequence to obtain a decrypted image sequence; the step of sequentially determining the action score of each image after the starting image in the image sequence and the action score of the current action content includes: sequentially determining the action score of each image after the starting image in the decrypted image sequence and the action score of the current action content.

[0070] In one possible implementation, determining the matching result between the image sequence and the action sequence further includes: generating at least one of facial region coordinates and facial number corresponding to the images in the image sequence; and determining the matching result based on the facial region coordinates, the facial number, the image sequence, and the action sequence.

[0071] In one possible implementation, determining the matching result based on at least one of the facial region coordinates and the facial number, as well as the image sequence and the action sequence, includes: determining the matching result between the facial region indicated by the facial region coordinates and the action sequence in each image of the image sequence, and using this as the matching result between the image sequence and the action sequence.

[0072] In one possible implementation, determining the matching result based on at least one of the facial region coordinates and the facial ID, as well as the image sequence and the action sequence, further includes: if the number of images corresponding to the least frequent facial ID in the image sequence is greater than a first threshold, determining the matching result as a matching failure.

[0073] In one possible implementation, the detection device is further configured to perform at least one of the following: if the detection result is determined to be a detection failure, sending a first instruction to the terminal, the first instruction controlling the terminal to enter a page for resending the detection request; if the number of times the first instruction is sent to the terminal within a third preset time period reaches a second threshold, in response to a detection request sent by the terminal through the page, sending a second instruction to the terminal, the second instruction notifying the terminal that the server rejects the detection request; and if the time from sending the first instruction to receiving a new image sequence is determined to be greater than a fourth preset time, sending the second instruction to the terminal.

[0074] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0075] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium can be volatile or non-volatile.

[0076] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above-described method.

[0077] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0078] Figure 4 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. For example, the electronic device 1900 may be provided as a server. (Refer to...) Figure 4 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0079] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output (I / O) interface 1958. Electronic device 1900 can operate on an operating system stored in memory 1932, such as Microsoft Server operating system (Windows Server). TM Apple's graphical user interface-based operating system (Mac OSX) TM ), a multi-user, multi-process computer operating system (Unix) TM Linux is a free and open-source Unix-like operating system. TM ), the open-source Unix-like operating system (FreeBSD) TM (or similar.)

[0080] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0081] The aforementioned electronic devices may be provided as terminals, servers, or other forms of devices.

[0082] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0083] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, (but not limited to) electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0084] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0085] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0086] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0087] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0088] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0089] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0090] The computer program product can be implemented through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK).

[0091] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A detection method applied to a server, characterized in that, The detection method includes: The receiving terminal responds to the image sequence sent by the action sequence, the image sequence including multiple frames of images; One action content from the action sequence is sequentially acquired as the current action content, and the following operations are performed: determining the starting image corresponding to the current action content; sequentially determining the action scores of each image after the starting image in the image sequence and the current action content; determining the matching result corresponding to the current action content based on the action score of any image; and determining the matching result between the image sequence and the action sequence based on the matching results corresponding to all action content in the action sequence. Based on the matching results between the image sequence and the action sequence, a detection result is generated; The step of generating detection results based on the matching results between the image sequence and the action sequence includes: If the matching result between the image sequence and the action sequence is determined to be a successful match, the first image in the image sequence is selected. Based on the first image, generate a liveness detection sub-result corresponding to the first image; The first image with the highest action score will be used as the second image. If the liveness detection sub-result corresponding to the second image is determined to be live, and the ratio of the number of first images with liveness detection sub-results to the total number of all first images is greater than or equal to a preset ratio, then the liveness detection result is determined to be live.

2. The detection method as described in claim 1, characterized in that, Determining the starting image corresponding to the current action content includes: when the current action content is determined to be the first action content in the action sequence, the starting image is the starting image of the image sequence; when the current action content is determined not to be the first action content in the action sequence, the starting image is the next frame image of the image that successfully matches the previous action content.

3. The detection method as described in claim 1 or 2, characterized in that, Determining the matching result between the image sequence and the action sequence includes: If no successful matching result is obtained for the action sequence within a first preset time period starting from the start of determining the matching result of the action sequence, and / or if no successful matching result is obtained for any action content within a second preset time period starting from the start of determining the matching result of any action content in the action sequence, the matching result of the image sequence and the action sequence is determined to be a failed match.

4. The detection method as described in claim 1 or 2, characterized in that, The step of generating detection results based on the matching results between the image sequence and the action sequence includes: Based on the liveness detection result, the detection result is determined, wherein if the liveness detection result indicates that the individual is alive, the detection result is deemed to be a successful detection.

5. The detection method as described in claim 4, characterized in that, The step of selecting the first image from the image sequence includes: Select a preset number of first images from the image sequence whose action scores are greater than or equal to a second score threshold.

6. The detection method as described in claim 1 or 2, characterized in that, The receiving terminal responds to the image sequence sent by the action sequence, including: The image sequence sent by the terminal in response to the action sequence is decrypted to obtain the decrypted image sequence; The step of sequentially determining the action scores of each image after the starting image in the image sequence and the current action content includes: sequentially determining the action scores of each image after the starting image in the decrypted image sequence and the current action content.

7. The detection method as described in claim 1 or 2, characterized in that, Determining the matching result between the image sequence and the action sequence further includes: Generate at least one of the facial region coordinates and facial number corresponding to the images in the image sequence; The matching result is determined based on at least one of the facial region coordinates and the facial number, as well as the image sequence and the action sequence.

8. The detection method as described in claim 7, characterized in that, Determining the matching result based on at least one of the facial region coordinates and the facial number, as well as the image sequence and the action sequence, includes: The matching result between the facial region indicated by the facial region coordinates and the action sequence in each image of the image sequence is determined as the matching result between the image sequence and the action sequence.

9. The detection method as described in claim 7, characterized in that, The step of determining the matching result based on at least one of the facial region coordinates and the facial number, as well as the image sequence and the action sequence, further includes: If the number of images corresponding to the face number with the fewest occurrences in the image sequence is greater than a first threshold, the matching result is determined to be a matching failure.

10. The detection method as described in claim 1 or 2, characterized in that, The detection method further includes at least one of the following: If the detection result is determined to be a failure, a first instruction is sent to the terminal, which controls the terminal to enter a page for resending the detection request; If the number of times the first instruction is sent to the terminal within a third preset time reaches a second threshold, in response to the detection request sent by the terminal through the page, a second instruction is sent to the terminal. The second instruction is used to notify the terminal that the server rejects the detection request. If the time from sending the first instruction to receiving the new image sequence is determined to be greater than a fourth preset time, a second instruction is sent to the terminal.

11. A face detection device, applied to a server, characterized in that, The detection device includes: An image sequence receiving module is used to receive an image sequence sent by the terminal in response to an action sequence, wherein the image sequence includes multiple frames of images; The action content processing module is used to sequentially acquire one action content from the action sequence as the current action content, and perform the following operations: determine the starting image corresponding to the current action content; sequentially determine the action scores of each image after the starting image in the image sequence and the current action content; determine the matching result corresponding to the current action content based on the action score of any image; and determine the matching result between the image sequence and the action sequence based on the matching results corresponding to all action content in the action sequence. The detection result generation module is used to generate detection results based on the matching results between the image sequence and the action sequence; The step of generating detection results based on the matching results between the image sequence and the action sequence includes: If the matching result between the image sequence and the action sequence is determined to be a successful match, the first image in the image sequence is selected. Based on the first image, generate a liveness detection sub-result corresponding to the first image; The first image with the highest action score will be used as the second image. If the liveness detection sub-result corresponding to the second image is determined to be live, and the ratio of the number of first images with liveness detection sub-results to the total number of all first images is greater than or equal to a preset ratio, then the liveness detection result is determined to be live.

12. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the detection method according to any one of claims 1 to 10.

13. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the detection method according to any one of claims 1 to 10.

Images