Light sequence detection method, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an optical sequence detection method, electronic equipment and a storage medium. The light sequence detection method comprises the following steps: acquiring a to-be-detected video of a to-be-detected object, wherein the to-be-detected video comprises a video acquired when the to-be-detected object is irradiated according to a detection illumination sequence; based on the detection illumination sequence, performing light sequence detection on the to-be-detected video to obtain a light sequence detection result of the to-be-detected video; if the light sequence detection result shows that the to-be-detected object does not pass the light sequence detection, detecting whether an acquisition environment of the to-be-detected video meets a light sequence detection condition or not; and if the acquisition environment of the to-be-detected video does not meet the light sequence detection condition, correcting the light sequence detection result of the to-be-detected video. The optical sequence detection failure caused by environmental reasons can be avoided, and the passing rate of optical sequence detection is improved on the premise of ensuring the accuracy of optical sequence detection. The light sequence detection method is higher in practicability and better in user experience.

Description

Technical Field

[0001] The present application relates to the field of image processing, and more specifically, to a light sequence detection method, electronic device, and storage medium. Background Art

[0002] With the development of image processing technology, light sequence detection is being applied in various application scenarios, such as identity verification. During light sequence detection, the emitted light sequence is compared with the sequence of reflected light presented by the target object in the video. If the two are consistent, the light sequence detection is considered to have passed; otherwise, the light sequence detection is considered to have failed.

[0003] However, the light sequence detection results are easily affected by the surrounding environment. For example, in backlit or outdoor environments, the colored light projected by the screen on the face will be greatly weakened, which can easily cause the light sequence detection to fail, thereby affecting the accuracy of the light sequence detection. Summary of the Invention

[0004] The present application has been made in view of the above-mentioned problems.

[0005] According to one aspect of the present application, a light sequence detection method is provided. The light sequence detection method includes:

[0006] Obtaining a video to be detected of the object to be detected, wherein the video to be detected includes a video captured when the object to be detected is illuminated according to a detection illumination sequence;

[0007] Based on the detection light sequence, the video to be detected is subjected to light sequence detection to obtain a light sequence detection result of the video to be detected;

[0008] If the optical sequence detection result indicates that the object to be detected has not passed the optical sequence detection, then the acquisition environment of the video to be detected is detected to see whether it meets the optical sequence detection conditions;

[0009] If the acquisition environment of the video to be detected does not meet the optical sequence detection conditions, the optical sequence detection result of the video to be detected is corrected.

[0010] According to another aspect of the present application, an electronic device is provided, including: a processor and a memory, wherein computer program instructions are stored in the memory, and the computer program instructions are used by the processor to execute the optical sequence detection method described above when the processor is executed.

[0011] According to another aspect of the present application, a storage medium is provided, on which program instructions are stored. When the program instructions are executed by a processor, they are used to execute the optical sequence detection method described above.

[0012] In the above technical solution, when the object to be detected fails the optical sequence detection, it is detected whether the acquisition environment of the video to be detected meets the optical sequence detection conditions. If the acquisition environment of the video to be detected does not meet the optical sequence detection conditions, the optical sequence detection result of the video to be detected is corrected. In this way, it is possible to avoid the situation where a video that should have passed the optical sequence detection fails to pass the optical sequence detection due to environmental reasons, thereby improving the pass rate of the optical sequence detection. The detection of whether the acquisition environment meets the optical sequence detection conditions is performed after the optical sequence detection, which can not only ensure the accuracy of the optical sequence detection, but also help to combine with other detection methods to realize specific detection functions, such as identity verification. This optical sequence detection method has higher practicality and better user experience.

[0013] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application.

[0015] Figure 1 FIG1 shows a schematic flow chart of an optical sequence detection method according to an embodiment of the present application;

[0016] Figure 2 A schematic flowchart of a training process of an image classification model according to an embodiment of the present application is shown;

[0017] Figure 3 FIG2 shows a schematic flow chart of an optical sequence detection method according to another embodiment of the present application;

[0018] Figure 4A shows a schematic diagram of target image processing according to one embodiment of the present application;

[0019] Figure 4B FIG2 shows a schematic diagram of target image processing according to another embodiment of the present application;

[0020] Figure 5 A flow chart of a light sequence detection method according to another embodiment of the present application is shown;

[0021] Figure 6 shows a schematic block diagram of an optical sequence detection device according to an embodiment of the present application;

[0022] Figure 7A schematic block diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] It should be noted that the videos, images, test results and other data obtained by this application scheme are accessed, collected, stored and used for subsequent analysis and processing with the consent and authorization of the user or the relevant data owner after the user or the relevant data owner is clearly informed of the data collection content, data purpose, processing method and other information, and the user or the relevant data owner can be provided with ways to access, correct and delete the data, as well as methods to revoke consent and authorization.

[0024] In order to make the purpose, technical solutions and advantages of this application more apparent, the following will describe in detail exemplary embodiments of this application with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this application.

[0025] In order to at least solve the above technical problems, the present application proposes an optical sequence detection method. Figure 1 FIG1 shows a schematic flow chart of an optical sequence detection method according to an embodiment of the present application. The detection method can be executed independently by a terminal device or by a background server. Figure 1 As shown, the optical sequence detection method includes the following steps S1100 to S1400.

[0026] In step S1100, a video to be detected of the object to be detected is obtained, wherein the video to be detected includes a video captured when the object to be detected is illuminated according to a detection illumination sequence.

[0027] In step S1200 , based on the detection light sequence, light sequence detection is performed on the video to be detected to obtain a light sequence detection result of the video to be detected.

[0028] In step S1300 , if the optical sequence detection result indicates that the object to be detected fails the optical sequence detection, it is detected whether the acquisition environment of the video to be detected meets the optical sequence detection conditions.

[0029] In step S1400 , if the acquisition environment of the video to be detected does not meet the optical sequence detection condition, the optical sequence detection result of the video to be detected is corrected.

[0030] Exemplarily, in step S1100, a video to be detected of the object to be detected is acquired. The video to be detected can be captured by a terminal device. For example, the terminal device can receive a detection illumination sequence sent by a backend server or generate the detection illumination sequence itself. The terminal device can illuminate the object to be detected according to the detection illumination sequence and capture a video as the video to be detected. In some embodiments, the optical sequence detection method can be executed by a backend server. After the terminal device captures the video to be detected, the backend server can obtain the video to be detected from the terminal device, and the backend server can execute subsequent steps. In other embodiments, the optical sequence detection method can be executed by the terminal device, and after the terminal device captures the video to be detected, i.e., after obtaining the video to be detected, the terminal device can execute subsequent steps. A segment of the video to be detected can include a portion of the object to be detected when it is not illuminated by the detection illumination sequence and a portion of the object to be detected when it is illuminated by the detection illumination sequence. In other words, a segment of the video to be detected can include multiple images, i.e., video frames. Portions of these video frames can be captured when the object to be detected is not illuminated by the detection illumination sequence. Another portion of these video frames may be captured while the object to be inspected is being illuminated by the detection illumination sequence. The latter portion may include, for example, images captured before or after the object to be inspected is illuminated by the detection illumination sequence, and images captured during the time period between the time periods of emitting different detection illumination sequences.

[0031] In step S1200, based on the detection light sequence, light sequence detection is performed on the video to be detected to obtain a light sequence detection result of the video to be detected. Through light sequence detection, it can be determined whether the reflection condition of the object to be detected in the video to be detected when it is illuminated matches the detection light sequence. If it matches, the light sequence detection result indicates that the object to be detected has passed the light sequence detection; if it does not match, the light sequence detection result indicates that the object to be detected has not passed the light sequence detection. In an embodiment of the present application, step S1200 can be implemented using any existing or future developed light sequence detection algorithm. Through light sequence detection, it can be determined whether the camera has been hijacked, and it has a high defense capability against injection attacks, high-definition screen reshoot attacks and other attack methods.

[0032] Exemplarily, the light sequence detection result may include the matching confidence of the detected light sequence and the video to be detected. If the matching confidence is greater than or equal to the first confidence threshold, it indicates that the light sequence detection has passed; if the matching confidence is less than the first confidence threshold, it indicates that the light sequence detection has not passed. In a specific embodiment, the above-mentioned first confidence threshold is 0.5. It can be understood that the above-mentioned first confidence threshold can be set according to the needs of the application scenario. If the requirements for light sequence detection are more stringent, the above-mentioned first confidence threshold can be set higher; otherwise, vice versa. The above-mentioned first confidence threshold can be any value between 0.3 and 0.7.

[0033] In step S1300 , if the optical sequence detection result indicates that the object to be detected fails the optical sequence detection, it is detected whether the acquisition environment of the video to be detected meets the optical sequence detection conditions.

[0034] If the optical sequence detection result indicates that the object to be inspected has failed the optical sequence detection, it may be a black market attack. It may also fail the optical sequence detection due to environmental factors. For example, the video to be inspected may be backlit or captured outdoors. In this case, the clarity of the reflected light in the video to be inspected is poor, resulting in failure of the optical sequence detection. In other words, due to the negative influence of the environment in which the video to be inspected was captured, a normal object to be inspected may be judged as a black market attack.

[0035] In order to avoid normal objects to be inspected from being judged as black market attack behaviors and to improve the real person pass rate, when the light sequence detection result indicates that the object to be inspected has not passed the light sequence detection, the acquisition environment of the video to be inspected can be detected to determine whether the acquisition environment meets the light sequence detection conditions. In some embodiments, the brightness, contrast, saturation and other parameters of the video to be inspected can be detected, and the situation of multiple parameters can be comprehensively considered to jointly determine whether the acquisition environment meets the light sequence detection requirements. For example, if the brightness of the video to be inspected is high and the contrast is low, it can be determined that the acquisition environment of the video to be inspected does not meet the light sequence detection conditions. In other embodiments, a frame of image can be extracted from the video to be inspected, and the image can be detected to determine whether the acquisition environment meets the light sequence detection conditions. For example, an existing or future developed deep learning model can be used to detect the image. For example, using an image classification model, the categories can include meeting the light sequence detection conditions and not meeting the light sequence detection conditions. By extracting image features, the image is classified. If the image is classified as meeting the light sequence detection requirements, the corresponding acquisition environment of the video to be detected meets the light sequence detection requirements; if the image is classified as not meeting the light sequence detection requirements, the corresponding acquisition environment of the video to be detected does not meet the light sequence detection requirements.

[0036] In step S1400 , if the acquisition environment of the video to be detected does not meet the optical sequence detection condition, the optical sequence detection result of the video to be detected is corrected.

[0037] As previously described in step S1300, step S1300 is executed when the optical sequence detection result indicates that the object to be inspected has failed the optical sequence detection. Under this premise, if the capture environment of the video to be inspected does not meet the optical sequence detection conditions, the corresponding video to be inspected may have failed the optical sequence detection due to environmental influences. Therefore, the initial optical sequence detection failure of the video to be inspected obtained in step S1200 can be corrected. This ensures the practicality of optical sequence detection.

[0038] In the above technical solution, when the object to be detected fails the optical sequence detection, it is detected whether the acquisition environment of the video to be detected meets the optical sequence detection conditions. If the acquisition environment of the video to be detected does not meet the optical sequence detection conditions, the optical sequence detection result of the video to be detected is corrected. In this way, it is possible to avoid the situation where a video that should have passed the optical sequence detection fails to pass the optical sequence detection due to environmental reasons, thereby improving the pass rate of the optical sequence detection. The detection of whether the acquisition environment meets the optical sequence detection conditions is performed after the optical sequence detection, which can not only ensure the accuracy of the optical sequence detection, but also help to combine with other detection methods to realize specific detection functions, such as identity verification. This optical sequence detection method has higher practicality and better user experience.

[0039] Exemplarily, step S1400 corrects the optical sequence detection result of the video to be detected, including step S1410 or step S1420.

[0040] In some embodiments, step S1410 determines the optical sequence detection result of the video to be detected as having passed the optical sequence detection. It is understood that if the optical sequence detection result of the video to be detected indicates that it has failed the optical sequence detection, but does not meet the optical sequence detection conditions, the optical sequence detection result of the video to be detected may be determined to have failed the optical sequence detection.

[0041] In other embodiments, step S1420 removes the video to be detected from the video sequence that has not passed the optical sequence detection. For example, the video sequence can be a list of stored data, which can have a video sequence that has passed the optical sequence detection and a video sequence that has not passed the optical sequence detection. After the optical sequence detection result of the video to be detected is determined in step S1200, the video to be detected can be stored in the video sequence that has passed the optical sequence detection or the video sequence that has not passed the optical sequence detection. In the case where the optical sequence detection result of the video to be detected indicates that it has not passed the optical sequence detection but does not meet the optical sequence detection conditions, the video to be detected can be removed from the video sequence that has not passed the optical sequence detection. For example, after removal, it can be stored in a correction sequence to distinguish it from the video to be detected that has passed the optical sequence detection or the video that has not passed the optical sequence detection.

[0042] The above technical solution determines the optical sequence detection result of the video to be tested as having passed the optical sequence detection, which can improve the pass rate of the optical sequence detection. Removing the video to be tested from the video sequence that has not passed the optical sequence detection can also improve the pass rate of the optical sequence detection and enable differentiation, providing a basis for subsequent operations.

[0043] Exemplarily, step S1300 detects whether the acquisition environment of the video to be detected meets the light sequence detection condition, including: step S1310 and step S1320.

[0044] In step S1310, a frame of target image is selected from the video to be detected. The target image is an image captured when the object to be detected is not illuminated by the detection illumination sequence. The target image captured when the object to be detected is not illuminated by the detection illumination sequence is determined in the video to be detected. In some embodiments, the video to be detected can be captured before the detection illumination sequence begins to illuminate the object to be detected. In other words, the front part of the video to be detected can include the target image captured when the object to be detected is not illuminated by the detection illumination sequence. In other embodiments, the video to be detected can continue to be captured after the object to be detected is illuminated based on the detection illumination sequence. In other words, the back part of the video to be detected can include the target image captured when the object to be detected is not illuminated by the detection illumination sequence. It is understood that the target image captured when the object to be detected is not illuminated by the detection illumination sequence can also be the middle part of the video to be detected. The specific position of the target image in the video to be detected can be determined based on the specific method of capturing the video to be detected. Any frame of image can be determined as the target image among all the videos captured when the object to be detected is not illuminated by the detection illumination sequence. Preferably, the clearest frame of image can be determined as the target image from all videos captured when not illuminated by the detection illumination sequence. Alternatively, multiple frames of image can be determined as the target images from all videos captured when not illuminated by the detection illumination sequence in the video to be detected.

[0045] In step S1320, the target image is classified using an image classification model to determine whether the acquisition environment of the video to be detected corresponding to the target image meets the light sequence detection condition. The image classification model can be used to classify whether the target image meets the light sequence detection condition.

[0046] The image classification model may include a backbone network and a classification layer. The backbone network of the image classification model may include ResNet, MobileNet, ShuffleNet, EffNet, Vit, Swin-Transformer, etc. Optionally, the corresponding backbone network can be determined based on the computing power of the deployment scenario. For example, when running on a smart mobile device or a central processing unit (CPU), a lightweight backbone network with low computing power requirements, such as MobileNet, ShuffleNet, EfficientNet, etc., can be selected. When running on a graphics processing unit (GPU) or a neural network processor (NPU), a backbone network with higher computing power requirements can be selected to improve classification accuracy. The classification layer of the image classification model can be of any form, such as a linear layer or a fully connected layer.

[0047] In some embodiments, the image classification model can be a single-input model, that is, one target image is input to obtain a classification result. In other embodiments, the image classification model can be a multi-input model, which simultaneously inputs multiple target images to obtain a classification result. The classification result indicates whether the input target image meets the light sequence detection condition. The classification result may include the confidence level that the target image meets the light sequence detection condition. For example, if the confidence level is greater than or equal to the second confidence threshold, the classification result indicates that the target image meets the light sequence detection condition; if the confidence level is less than the second confidence threshold, the classification result indicates that the target image does not meet the light sequence detection condition. In a specific embodiment, the second confidence threshold is 0.5. It can be understood that the second confidence threshold can be set according to the needs of the application scenario. If the light sequence detection result is more relied upon and trusted, the second confidence threshold can be set lower; otherwise, vice versa. The second confidence threshold can be any value between 0.3 and 0.7.

[0048] Optionally, the image classification model can be used for zero-shot classification without training. Alternatively, the image classification model can be obtained through training to improve its classification accuracy.

[0049] In the case where the object to be inspected fails the light sequence detection, the above technical solution identifies a target image captured in the video to be inspected when the object was not illuminated by the detection light sequence, and uses the image classification model to classify the target image. The image classification model classifies the target image with high accuracy and robustness, and has strong generalization capabilities for different objects to be inspected. This improves the accuracy of determining whether the acquisition environment of the video to be inspected meets the light sequence detection requirements.

[0050] For example, Figure 2 FIG1 shows a schematic flow chart of the training process of an image classification model according to an embodiment of the present application. Figure 2 As shown, the image classification model is obtained through the training process of step S2100, step S2200 and step S2300.

[0051] In step S2100 , light sequence detection is performed on training videos collected in different environments to obtain a light sequence detection result for each training video, wherein the different environments include at least one of different time periods, different lighting conditions, and / or different locations.

[0052] The training videos include videos captured under different environments while illuminating the target object according to the training lighting sequence. For example, different time periods can include early morning, morning, noon, afternoon, evening, and nighttime. Different locations can include indoor and outdoor locations. Different lighting conditions can include different brightness levels, backlighting, front lighting, side lighting, and other different acquisition angles. By combining these different conditions, a variety of different acquisition environments can be determined. Training videos can then be captured under different acquisition environments while illuminating the target object according to the training lighting sequence. Multiple training videos can be captured by combining different environmental conditions. Furthermore, training videos can be captured for different target objects. Preferably, additional training videos can be captured for environments known to not meet the light sequence detection conditions. For example, additional training videos can be captured in backlit and outdoor environments. The different target objects may also affect whether the light sequence detection conditions are met. For example, the whiteness and oiliness of the target object's face can affect their reflectivity, thereby affecting the light sequence detection results.

[0053] Light sequence detection can be performed on each training video to obtain a light sequence detection result for each training video. The light sequence detection result can indicate whether the training video passed or failed the light sequence detection. In some embodiments, when illuminating the target object with a training illumination sequence to capture the video, the same or different training illumination sequences can be used. When different training illumination sequences are used, the training illumination sequence corresponding to each training video can be recorded. Light sequence detection can be performed on each training video based on the training illumination sequence corresponding to each training video. For example, the light sequence detection result can include a light sequence quality score. If the light sequence quality score of the training video is greater than a score threshold, it indicates that the training video meets the light sequence detection requirements; otherwise, it indicates that the training video does not meet the light sequence detection requirements. In a specific embodiment, the score threshold is 0.5. It will be understood that the score threshold can be set according to the needs of the application scenario. If the light sequence detection results are more stringent, the score threshold can be set higher; otherwise, it can be set lower. The score threshold can be any value between 0.3 and 0.7.

[0054] In step S2200, training samples are generated based on each training video and the corresponding light sequence detection results. The training samples include training images and corresponding labels. The training images are images of the target object in the training video that are captured when the training light sequence is not illuminated by the training light sequence. If the light sequence detection result of the training video indicates that the light sequence detection is passed, the label corresponding to the training image in the training video is "satisfying the light sequence detection condition." If the light sequence detection result of the training video indicates that the light sequence detection is failed, the label corresponding to the training image in the training video is "not satisfying the light sequence detection condition."

[0055] Images captured when the target object was not illuminated by the training illumination sequence can be determined in the training video. In some embodiments, images captured when the target object was not illuminated by the training illumination sequence can be determined based on the acquisition of the training images. For example, acquisition of the training video can begin before the target object is illuminated by the training illumination sequence. The training images can be determined at the beginning of the training video. Optionally, the training images can include at least one frame of an image in a training video that was not illuminated by the training illumination sequence. This can prevent the illumination of the training illumination sequence from negatively impacting image information and hindering accurate image classification.

[0056] The labels corresponding to the training images can be determined based on the light sequence detection results of the training video. It can be understood that whether a training video passes the light sequence detection can be determined by whether the environment in which the training video was captured meets the light sequence detection requirements. If the light sequence detection result of the training video indicates that it passes the light sequence detection, the labels corresponding to the training images in the training video are "meeting the light sequence detection conditions." If the light sequence detection result of the training video indicates that it fails the light sequence detection, the labels corresponding to the training images in the training video are "not meeting the light sequence detection conditions."

[0057] Thus, a training sample can be generated based on the training images and corresponding labels in the plurality of training videos. In other words, each training image in the training sample has a label indicating whether it satisfies the light sequence detection condition or not.

[0058] In step S2300, the training samples are input into the initial classification model, and the initial classification model is trained according to the output results to obtain an image classification model.

[0059] Training images are images of target objects captured in different environments. In theory, these images should all pass optical sequence detection, but may fail due to environmental influences. The training samples can be input into an initial classification model, and supervised training of the initial classification model is performed based on the output results to obtain an image classification model. The labels of the training images in the training samples can be used as supervisory data. The parameters of the initial classification model are adjusted based on the classification results and labels of the training images obtained by the initial classification model. For example, an arbitrary loss function can be used to adjust the parameters of the initial classification model to improve the classification accuracy of the training images, thereby obtaining an image classification model.

[0060] The above technical solution performs light sequence detection on training videos collected in different environments, generates training samples based on each training video and the corresponding light sequence detection results, and trains the initial classification model based on the training samples to obtain an image classification model. This allows accurate determination of which training images meet the light sequence detection conditions and which do not, allowing supervised training of the image classification model to improve the classification accuracy of the image classification model, enhance the accuracy of light sequence detection, and enhance the user experience.

[0061] Exemplarily, before classifying the target image using the image classification model, and after training the initial classification model at least once using training samples, the optical sequence detection method may further include performing a testing operation on the image classification model to test whether the classification performance of the current image classification model meets the requirements. In other words, determining through testing whether it can be used to classify the target image. Figure 3 FIG. 1 shows a schematic flow chart of an optical sequence detection method according to another embodiment of the present application. Figure 3 As shown, the light sequence detection method further includes step S3100 and step S3200. Step S3100 and step S3200 can be performed before the target image is classified using the image classification model and after the initial classification model is trained at least once using the training samples.

[0062] In step S3100, the image classification model is tested using a test sample to obtain a test result, wherein the test sample includes an attack image, and the attack image corresponds to a label that satisfies the light sequence detection condition.

[0063] A light sequence detection can be performed on the test video based on the test illumination sequence to obtain a light sequence detection result for each test video. In some embodiments, the test video can include videos captured under different environments using the test illumination sequence to illuminate different target objects. A light sequence detection can be performed on the test video. If the light sequence detection result for the test video indicates that the light sequence detection passes, then the capture environment corresponding to the test video meets the light sequence detection conditions; if the light sequence detection result for the test video indicates that the light sequence detection fails, then the capture environment corresponding to the test video does not meet the light sequence detection conditions.

[0064] A test sample can be generated based on each test video and the corresponding light sequence detection result. The test sample includes a test image and a corresponding label. The test image is an image captured in the test video when the target object is not illuminated by the test light sequence. If the light sequence detection result of the test video indicates that the light sequence detection condition is passed, the label corresponding to the test image in the test video is that the light sequence detection condition is met. If the light sequence detection result of the test video indicates that the light sequence detection condition is failed, the label corresponding to the test image in the test video is that the light sequence detection condition is not met. In some embodiments, after training the image classification model, a trained image classification model can be obtained. Image classification models that have undergone different training operations can have different parameters and thus different image classification performance. The image classification model can be tested using the test sample. The trained image classification model can be tested based on the light sequence detection results and test images of each test video. If the test image is an image captured in the test video when the target object is not illuminated by the test light sequence, theoretically, all test images of these target objects should pass the light sequence detection, but may fail due to environmental influences. Therefore, if the light sequence detection result of the test video indicates that it passes the light sequence detection, the classification result of the image classification model for the test image should be that it meets the light sequence detection conditions; if the light sequence detection result of the test video indicates that it fails the light sequence detection, the classification result of the image classification model for the test image should be that it does not meet the light sequence detection conditions.

[0065] The test samples also include attack images, with the corresponding label for the attack images being "satisfying the light sequence detection condition." Attack images are images that should not pass the light sequence detection. The label for the attack images is set to "satisfy the light sequence detection condition." In other words, the image classification model should classify the attack images as meeting the light sequence detection condition to avoid modifying the light sequence detection results. Optionally, black market attack images can be collected to serve as part of the test images. Alternatively, some attack images can be generated through simulation to serve as part of the test images.

[0066] By testing an image classification model with attack images, you can verify the model's security. For example, you can determine the probability that an attack image will be classified as not meeting the light order detection criteria to assess the model's security. The lower the probability that an attack image will be classified as not meeting the light order detection criteria, the higher the security of the image classification model. The performance of the image classification model can be comprehensively evaluated based on its accuracy in classifying non-attack images in the test image and its security in classifying attack images.

[0067] In step S3200, it is determined based on the test results whether the image classification model can be used to classify the target image.

[0068] The performance of the image classification model after different training operations can be determined based on the test results. For example, the test results can represent the accuracy of the image classification model in classifying the test image. A trained image classification model with an accuracy greater than an accuracy threshold, for example, an accuracy greater than 85%, can be determined as the image classification model used to classify the target image in step S1320. If the test results of the current image classification model do not meet the requirements, the next training operation can be performed again.

[0069] The above technical solution tests the image classification model using test samples, including attack images with labels corresponding to those that meet the light sequence detection requirements. The test results then determine whether the image classification model can be used to classify the target image. This ensures the performance of the image classification model used to classify the target image, thereby improving the accuracy of light sequence detection and reducing the probability of attack behaviors being modified to pass light sequence detection, thereby enhancing the security of light sequence detection and improving the user experience.

[0070] Exemplarily, step S1320 classifies the target image using an image classification model, including: step S1321 and step S1322.

[0071] In step S1321 : performing a plurality of different cropping and / or scaling operations on the target image to obtain a plurality of adjusted images corresponding to the target image.

[0072] After identifying a target image frame in the video to be detected that is not illuminated by the illumination sequence, the target image may contain a large amount of content, including not only the target object but also a large amount of background area. To prevent interference from the background area with the light sequence detection, the target image can be cropped to obtain an adjusted image. It can be understood that in this adjusted image, the target object is more prominent, facilitating light sequence detection.

[0073] In addition, the size of the target image may not meet the size requirements of the image classification model. Therefore, the size of the adjusted image can be made to meet the requirements of the image classification model by performing different cropping and / or scaling operations on the target image. In other words, the sizes of multiple adjusted images can be the same. Multiple adjusted images can be obtained by cropping different areas in the target image and scaling to different ratios. The contents of multiple adjusted images can be different. For example, in adjusted image 1, the face of the object to be detected may account for a higher proportion, while in adjusted image 2, the surrounding environment of the object to be detected may account for a higher proportion than in adjusted image 1.

[0074] In step S1420, the plurality of adjusted images are simultaneously input into the image classification model, and the image classification model outputs a classification result, which indicates whether the acquisition environment of the target image corresponding to the video to be detected meets the light sequence detection condition.

[0075] The image classification model can be a multi-input single-output model, in which multiple adjusted images of the target image are simultaneously input into the image classification model to obtain a classification result. The multiple adjusted images can have different features of the target image, and the image classification model can more accurately extract the features of the target image for classification. For example, the image classification model can extract features from different adjusted images separately, and can fuse the features of each adjusted image, for example, by directly aggregating the features of all adjusted images using the maximum pooling method, or can also splice the features of all adjusted images to achieve the fusion of the features of each adjusted image, obtain the fused features, and output the classification results based on the fused features to determine whether the acquisition environment of the target image corresponding to the video to be detected meets the light sequence detection conditions.

[0076] Optionally, the training images used for training the image classification model and the test images used for testing the image classification model can be subjected to the aforementioned various cropping and / or scaling operations to obtain multiple adjusted images corresponding to the training images or test images, respectively. The adjusted images are then used to train or test the image classification model.

[0077] The above technical solution performs various cropping and / or scaling operations on a target image to obtain multiple adjusted images corresponding to the target image. These images are then simultaneously input into an image classification model to obtain classification results. This allows the image classification model to capture more features of the target image, thereby improving classification accuracy and enhancing the user experience.

[0078] Exemplarily, step S1321: performing a plurality of different cropping and / or scaling operations on the target image to obtain a plurality of adjusted images corresponding to the target image, includes steps S1321A and / or S1321B. It is understood that there is no specific order in which steps S1321A and S1321B may be performed independently of each other or not.

[0079] In step S1321A: facial key points are recognized on the target image, and the target image is cropped based on the recognized facial key points, wherein the cropped image includes the entire facial area where the facial key points are located.

[0080] The target image can be subjected to facial key point recognition to determine the location of the face of the subject to be detected in the target image. After determining the location of the face of the subject to be detected in the target image, the target image can be cropped according to the size requirements of the image classification model. Optionally, the cropped image is a square. The cropped image includes the facial area of ​​the subject to be detected. It will be understood that the above cropping method can enable the image classification model to extract more facial features and avoid the influence of the environment in the target image on the facial features. Figure 4A FIG. 1 shows a schematic diagram of target image processing according to an embodiment of the present application. Figure 4A As shown in the figure, the white part in the target image is the object to be detected, and the circular area in the white part is the facial area of ​​the object to be detected. The white circular area in the processed image is the facial area of ​​the object to be detected. The facial area of ​​the object to be detected in the processed image is the main content of the image.

[0081] In step S1321B: face recognition is performed on the target image; based on the recognized face area, the target image is cropped along at least one long side thereof so that the recognized face in the cropped image is located on the perpendicular bisector of the short side of the cropped image; the cropped image is scaled so that the short side is scaled to a first preset size; the scaled image is cropped along its long side so that the long side conforms to a second preset size and retains the face area therein.

[0082] By performing face recognition on a target image, the location of a facial region in the target image can be determined. The target image can be cropped along at least one of its long sides, centered around the facial region. This cropping operation can position the identified facial region on the perpendicular bisector of the short side of the cropped image. For example, the facial region can be positioned slightly to the left of the target image. The target image can be cropped along its right long side, thereby removing a portion of the background area to the right of the facial region and aligning the facial region laterally with the center of the cropped image. The cropped image can be scaled to resize its short side to a first preset size. The first preset size can be a size required by an image classification model. The scaled image can then be cropped along its long side to ensure that the long side of the scaled image conforms to a second preset size. Optionally, the first preset size is equal to the second preset size. When cropping along the long side, at least the facial region is retained in the cropped image. Figure 4B FIG. 1 shows a schematic diagram of target image processing according to another embodiment of the present application. Figure 4BAs shown, the white portion in the target image is the object to be detected, and the circular area in the white portion is the face area of ​​the object to be detected. The white area in the processed image is the object to be detected. Compared with the processing in step S1411, the image processed in step S1412 not only includes more information about the object to be detected, but also includes more background area.

[0083] The above-described technical solution, including the cropping method in step S1321A that retains a larger facial area and the scaling and cropping method in step S1321B that retains more image information of the object to be detected, enables the image classification model to focus on the object to be detected, thereby achieving more accurate classification results for the object to be detected. Furthermore, the technical solution including both steps S1321A and S132B enables the image classification model to more accurately extract the different features of the object to be detected, further improving the accuracy of its classification results and, in turn, the accuracy of light sequence detection.

[0084] Exemplarily, the optical sequence detection method further includes step S1500 and step S1600. In step S1500: for the object to be detected, liveness detection is performed using the video to be detected to obtain a preliminary liveness detection result.

[0085] Existing or future liveness detection methods can be used to perform liveness detection on the subject using the video to be detected to obtain a preliminary liveness detection result. The preliminary liveness detection result can indicate whether the subject to be detected is a real person. The liveness detection in step S1500 and the light sequence detection in step S1200 can be performed independently or simultaneously after obtaining the video to be detected.

[0086] In step S1600 , a final liveness detection result of the object to be detected is determined according to the light sequence detection result and the preliminary liveness detection result.

[0087] In the aforementioned optical sequence detection method, the optical sequence detection result of the video to be detected can be obtained in step S1200. It can be understood that the optical sequence detection result of some cases is corrected in step S1400 to update the optical sequence detection result.

[0088] In some embodiments, the light sequence detection results and the preliminary liveness detection results can be represented by a probability value of passing the detection. Based on the light sequence detection results and the preliminary liveness detection results, the probability of passing the final liveness detection can be determined as the final liveness detection result. In other embodiments, the light sequence detection results and the preliminary liveness detection results can be represented by their respective confidence levels, and the final liveness detection result can be determined based on the confidence levels. This final liveness detection result can be used in application scenarios such as identity verification.

[0089] The above technical solution determines the final liveness detection result of the subject to be inspected based on the light sequence detection results and the preliminary liveness detection results. Combining the light sequence detection results with the preliminary liveness detection results can comprehensively determine whether the subject can pass the inspection, improving the accuracy of liveness detection, thereby enhancing the security of identity verification and improving the user experience.

[0090] Figure 5 FIG. 4 shows a flow chart of an optical sequence detection method according to another embodiment of the present application. Figure 5 As shown, after obtaining the video to be detected, light sequence detection is performed, and the light sequence detection result can be the matching confidence of the illumination sequence and the video to be detected. If the matching confidence is greater than or equal to 0.5, the light sequence detection is passed. If the matching confidence is less than 0.5, quality inspection frame extraction is performed in the video to be detected, that is, the clearest frame of image that is not illuminated by the illumination sequence is determined as the target image, and the target image is preprocessed. The preprocessing can be different cropping and / or scaling of the target image, and multiple preprocessed target images are input into the image classification model at the same time to obtain the classification result. The classification result can be the confidence indicating that the target image meets the light sequence detection conditions. If the confidence is greater than or equal to 0.5, the light sequence detection is not passed. If the confidence is less than 0.5, the light sequence detection is passed.

[0091] Illustratively, according to another aspect of the present application, an optical sequence detection device is also provided. Figure 6 FIG. 6 shows a schematic block diagram of an optical sequence detection device 600 according to an embodiment of the present application. Figure 6 As shown, the optical sequence detection device 600 includes an acquisition module 610 , a first detection module 620 , a second detection module 630 and a correction module 640 .

[0092] The acquisition module 610 is used to acquire a video to be detected of the object to be detected, wherein the video to be detected includes a video captured when the object to be detected is illuminated according to a detection illumination sequence. The first detection module 620 is used to perform optical sequence detection on the video to be detected based on the detection illumination sequence to obtain an optical sequence detection result for the video to be detected. The second detection module 630 is used to detect whether the acquisition environment of the video to be detected meets the optical sequence detection conditions if the optical sequence detection result indicates that the object to be detected has failed the optical sequence detection. The correction module 640 is used to correct the optical sequence detection result of the video to be detected if the acquisition environment of the video to be detected does not meet the optical sequence detection conditions.

[0093] Exemplarily, the correction module 640 includes a first correction submodule or a second correction submodule. The first correction submodule is used to determine the optical sequence detection result of the video to be detected as passing the optical sequence detection. The second correction submodule is used to remove the video to be detected from the video sequence that does not pass the optical sequence detection.

[0094] Exemplarily, the second detection module 630 includes a first determination submodule and a classification submodule. The first determination submodule is configured to select a target image frame from the video to be detected. The target image is an image captured when the object to be detected is not illuminated by the detection illumination sequence. The classification submodule is configured to classify the target image using an image classification model to determine whether the acquisition environment of the video to be detected corresponding to the target image meets the light sequence detection conditions.

[0095] Exemplarily, the light sequence detection device 600 further includes a training module. The training module includes a first detection submodule, a first sample submodule, and a training submodule. The first detection submodule is configured to perform light sequence detection on training videos captured in different environments to obtain light sequence detection results for each training video, wherein different environments include at least one of different time periods, different lighting conditions, and / or different locations. The first sample submodule is configured to generate training samples based on each training video and the corresponding light sequence detection results. The training samples include training images and corresponding labels. The training images are images of the target object in the training video captured when the training light sequence is not illuminated by the training light sequence. If the light sequence detection result of the training video indicates that the light sequence detection is passed, the label corresponding to the training image in the training video is that the light sequence detection condition is met. If the light sequence detection result of the training video indicates that the light sequence detection is failed, the label corresponding to the training image in the training video is that the light sequence detection condition is not met. The training submodule is configured to input the training samples into an initial classification model and train the initial classification model based on the output results to obtain an image classification model.

[0096] Exemplarily, before classifying the target image using the image classification model, and after training the initial classification model at least once using training samples, the light sequence detection device 600 further includes a testing module and a first determination module. The testing module is configured to test the image classification model using test samples to obtain test results, wherein the test samples include attack images corresponding to labels that satisfy light sequence detection conditions. The first determination module is configured to determine, based on the test results, whether the image classification model can be used to classify the target image.

[0097] Exemplarily, the classification submodule includes an adjustment unit and a classification unit. The adjustment unit is configured to perform various cropping and / or scaling operations on a target image to obtain multiple adjusted images corresponding to the target image. The classification unit is configured to simultaneously input the multiple adjusted images into an image classification model, which outputs a classification result indicating whether the acquisition environment of the target image corresponding to the video to be detected meets the light sequence detection conditions.

[0098] Exemplarily, the adjustment unit includes a first adjustment subunit and / or a second adjustment subunit. The first adjustment subunit is used to perform facial key point recognition on the target image and crop the target image based on the recognized facial key points, wherein the cropped image includes the entire facial area where the facial key points are located. The second adjustment subunit is used to perform facial recognition on the target image; crop the target image along at least one long side based on the recognized facial area so that the recognized facial area in the cropped image is located on the perpendicular bisector of the short side of the cropped image; scale the cropped image so that the short side of the cropped image is scaled to a first preset size; and crop the scaled image along its long side so that the long side meets a second preset size and retains the facial area therein.

[0099] Exemplarily, the optical sequence detection device 600 further includes a fourth detection module and a second determination module. The second detection module is configured to perform liveness detection on the subject to be detected using the video to be detected to obtain a preliminary liveness detection result. The second determination module is configured to determine a final liveness detection result for the subject to be detected based on the optical sequence detection result and the preliminary liveness detection result.

[0100] Illustratively, according to yet another aspect of the present application, an electronic device is also provided. Figure 7 A schematic block diagram of an electronic device 700 according to an embodiment of the present application is shown. The electronic device 700 includes a processor 710 and a memory 720. The memory 720 stores computer program instructions, which are used by the processor 710 to execute the optical sequence detection method described above when the computer program instructions are executed.

[0101] Illustratively, according to another aspect of the present application, a storage medium is further provided, wherein program instructions are stored on the storage medium, and the program instructions are used to execute the optical sequence detection method described above when running. The storage medium may include, for example, an erasable programmable read-only memory (EPROM), a portable read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The storage medium may be any combination of one or more computer-readable storage media.

[0102] Illustratively, according to another aspect of the present application, a computer program product is further provided, comprising computer program instructions, which are used to execute the optical sequence detection method described above when running.

[0103] A person skilled in the art can understand the specific implementation schemes and beneficial effects of the above-mentioned optical sequence detection device, electronic device, storage medium and computer program product by reading the above-mentioned description of the optical sequence detection method. For the sake of brevity, they will not be repeated here.

[0104] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely illustrative and are not intended to limit the scope of the present application. Various changes and modifications may be made therein by those skilled in the art without departing from the scope and spirit of the present application. All such changes and modifications are intended to be included within the scope of the present application as required by the appended claims.

[0105] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0106] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units described is merely a logical function division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another device, or ignoring or not performing some features.

[0107] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0108] Similarly, it should be understood that in order to streamline the present application and aid in understanding one or more of the various application aspects, in the description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, this approach of the present application should not be interpreted as reflecting the intention that the claimed application requires more features than those explicitly recited in each claim. More precisely, as reflected in the corresponding claims, the point of the application is that the corresponding technical problem can be solved with fewer features than all the features of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present application.

[0109] It will be understood by those skilled in the art that, except where mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus disclosed herein may be combined in any combination. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature providing the same, equivalent, or similar purpose.

[0110] Furthermore, those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, combinations of features from different embodiments are intended to be within the scope of this application and to form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.

[0111] The various component embodiments of the present application can be implemented in hardware, or implemented in a software module running on one or more processors, or implemented in a combination thereof. Those skilled in the art will appreciate that in practice, a microprocessor or a digital signal processor (DSP) can be used to implement some or all of the functions of some modules in the optical sequence detection device according to an embodiment of the present application. The application can also be implemented as a part or all of a device program (for example, a computer program and a computer program product) for performing the method described herein. Such a program realizing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0112] It should be noted that the above embodiments illustrate rather than limit the present application, and that a person skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbols placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0113] The above description is merely a specific embodiment or illustration of a specific embodiment of the present application, and the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included in the scope of protection of the present application. The scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A light sequence detection method, characterized in that: include: Acquire a video of the object to be inspected, wherein the video to be inspected includes a video captured when the object to be inspected is illuminated according to a detection illumination sequence; Based on the detection light sequence, performing light sequence detection on the video to be detected to obtain a light sequence detection result of the video to be detected; If the optical sequence detection result indicates that the object to be detected fails the optical sequence detection, detecting whether the acquisition environment of the video to be detected meets the optical sequence detection conditions; If the acquisition environment of the video to be detected does not meet the optical sequence detection condition, the optical sequence detection result of the video to be detected is corrected.

2. The optical sequence detection method according to claim 1, characterized in that: The correcting the optical sequence detection result of the video to be detected includes: Determining the optical sequence detection result of the video to be detected as passing the optical sequence detection; or The video to be detected is removed from the video sequence that has not passed the optical sequence detection.

3. The optical sequence detection method according to claim 1, wherein: The detecting whether the acquisition environment of the video to be detected meets the optical sequence detection condition includes: Selecting a frame of target image from the video to be detected, where the target image is an image captured when the object to be detected is not illuminated by the detection illumination sequence; The target image is classified using an image classification model to determine whether the acquisition environment of the to-be-detected video corresponding to the target image meets the light sequence detection condition.

4. The optical sequence detection method according to claim 3, characterized in that: The image classification model is trained through the following process: Performing light sequence detection on training videos collected under different environments to obtain a light sequence detection result for each training video, wherein the different environments include at least one of different time periods, different lighting conditions, and / or different locations; Generate training samples based on each training video and the corresponding light sequence detection result, wherein the training samples include training images and corresponding labels, and the training images are images in the training video captured when the target object is not illuminated by the training light sequence. If the light sequence detection result of the training video indicates that the light sequence detection is passed, then the label corresponding to the training image in the training video is that the light sequence detection condition is met; if the light sequence detection result of the training video indicates that the light sequence detection is not passed, then the label corresponding to the training image in the training video is that the light sequence detection condition is not met; The training samples are input into an initial classification model, and the initial classification model is trained according to the output results to obtain the image classification model.

5. The optical sequence detection method according to claim 4, characterized in that: Before classifying the target image using the image classification model and after training the initial classification model at least once using the training samples, the method further includes: Testing the image classification model using a test sample to obtain a test result, wherein the test sample includes an attack image, and the attack image corresponds to a label that satisfies a light sequence detection condition; Determine whether the image classification model can be used to classify the target image based on the test result.

6. The optical sequence detection method according to any one of claims 3 to 5, characterized in that: The classifying the target image by using an image classification model includes: performing a plurality of different cropping and / or scaling operations on the target image to obtain a plurality of adjusted images corresponding to the target image; Multiple adjusted images are simultaneously input into the image classification model so that the image classification model outputs a classification result, wherein the classification result indicates whether the acquisition environment of the target image corresponding to the video to be detected meets the light sequence detection condition.

7. The optical sequence detection method according to claim 6, characterized in that: The performing a plurality of different cropping and / or scaling operations on the target image to obtain a plurality of adjusted images corresponding to the target image includes: Recognize facial key points on the target image, and crop the target image based on the recognized facial key points to obtain the entire facial region including the facial key points; and / or Performing facial recognition on a target image; cropping the target image along at least one long side thereof based on the recognized facial region so that the recognized facial region in the cropped image is located on a perpendicular bisector of a short side of the cropped image; scaling the cropped image so that the short side of the cropped image is scaled to a first preset size; and cropping the scaled image along its long side so that the long side conforms to a second preset size and retains the facial region therein.

8. The optical sequence detection method according to claim 1, wherein: The method further comprises: For the object to be detected, performing liveness detection using the video to be detected to obtain a preliminary liveness detection result; A final liveness detection result of the object to be detected is determined according to the light sequence detection result and the preliminary liveness detection result.

9. An electronic device comprising: A processor and a memory, characterized in that The memory stores computer program instructions, which are used by the processor to execute the optical sequence detection method according to any one of claims 1 to 8 when the processor is running the computer program instructions.

10. A storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, they are used to execute the optical sequence detection method according to any one of claims 1 to 8.

Images