Method for detecting state of projector of structured light module, electronic device and storage medium

Abstract

Embodiments of the present application relate to the field of machine vision, and disclose a projector state detection method of a structured light module, an electronic device and a storage medium. An infrared camera of a structured light module is used to synchronously collect an infrared image and a speckle image, and a face frame and an eyeball key point in the speckle image are determined based on the infrared image, wherein the eyes of the person are not closed and are not blocked by an opaque object in the infrared image. In the case that it is detected that there is a speckle anomaly in the face frame, it is identified whether there is high-intensity reflected light at the eyeball key point. If there is, it is determined that the projector is in a strong light environment. The present scheme can effectively solve the problem of effectively obtaining the working state of the projector in the case that the ambient light is relatively strong in the face brushing recognition scene using the structured light module, and has low cost and does not need to additionally set a distance sensor.

Description

Technical Field

[0001] This invention relates to the field of machine vision, and in particular to a method for detecting the projector status of a structured light module, an electronic device, and a storage medium. Background Technology

[0002] Structured light modules have high requirements for both efficiency and security in facial recognition applications. The module obtains information about the working status of the structured light projector (hereinafter referred to as "projector"), such as whether the projector is blocked and whether the speckle pattern projected by the projector is normal. This plays an important role in the subsequent logic of the facial recognition application.

[0003] Existing technology uses a low-precision distance sensor near the projector for distance measurement. The sensor provides a relatively close distance value to infer whether the projector is obstructed. Additionally, when an object moves in front of the structured light module, the distance sensor detects this movement and, based on the coarse distance information, infers a suitable operating state for the projector. The high-precision 3D information obtained during the structured light module's operation is then used to verify the suitability of the projector's operating parameters.

[0004] However, when ambient light is strong, the energy of the speckle projected by the projector's rated power is submerged in the ambient light, making the speckle undetectable or of low quality in the image. The module cannot then determine whether the projector is blocked, not lit, or due to excessive ambient light. Furthermore, current market feedback on facial recognition modules indicates a strong demand for cost and size reduction. Some module products lack the budget and space to add a low-precision distance sensor, which places even higher demands on detecting the projector's operational status. Summary of the Invention

[0005] The purpose of this invention is to provide a projector state detection method, electronic device and storage medium for a structured light module, which can effectively solve the problem of effectively acquiring the working state of the projector in a face recognition scenario using a structured light module when the ambient light is strong, and is low in cost and does not require an additional distance sensor.

[0006] To address the aforementioned technical problems, embodiments of the present invention provide a projector state detection method for a structured light module, comprising:

[0007] The infrared camera of the structured light module synchronously acquires infrared images and speckle images, and determines the face frame and key points of the human eyeball in the speckle image based on the infrared image. In the infrared image, the human eye is not closed and is not blocked by an opaque object.

[0008] If abnormal speckle pattern is detected within the face frame, identify whether there is high-intensity reflected light at key points of the human eyeball;

[0009] If present, it is determined that the projector is in a strong light environment.

[0010] Embodiments of the present invention also provide an electronic device, comprising:

[0011] At least one processor; and,

[0012] A memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to perform the projector state detection method of the structured light module as described above.

[0014] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the projector state detection method of the structured light module as described above.

[0015] Compared to existing technologies, this invention utilizes an infrared camera within a structured light module to simultaneously acquire infrared and speckle images. Based on the infrared image, it determines the face frame and key points of the human eyeballs in the speckle image. In the infrared image, the human eyes are not closed and are not obstructed by opaque objects. If anomalies in the speckle pattern are detected within the face frame, the presence of high-intensity reflected light at the key points of the human eyeballs is identified. If present, it indicates that the projector is in a strong light environment. This solution leverages the fact that when the projector operates in a strong light environment, the eyes of the captured face act like convex lenses, reflecting light. This results in high-intensity reflections at the eyeball positions in the speckle image. Therefore, when anomalies in the face frame are detected, the presence of high-intensity reflected light at the key points of the human eyeballs in the speckle image can be used to determine whether the anomalies are caused by the projector being in a strong light environment. Attached Figure Description

[0016] Figure 1 This is a detailed flowchart of the projector state detection method for a structured light module according to an embodiment of the present invention. Figure 1 ;

[0017] Figure 2 This is a schematic diagram of a scene with high-intensity reflected light at key points of the human eyeball according to an embodiment of the present invention;

[0018] Figure 3 This is a detailed flowchart of the projector state detection method for a structured light module according to an embodiment of the present invention. Figure 2 ;

[0019] Figure 4 This is a schematic diagram of a scene with high-intensity reflected light in the area blocked by a light-transmitting object at the human eye, according to an embodiment of the present invention;

[0020] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been presented in the various embodiments of the present invention to enable the reader to better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments.

[0022] One embodiment of the present invention relates to a method for detecting the state of a projector in a structured light module. The method can be executed by the structured light module itself or by an electronic device with image processing capabilities connected to the structured light module. The method aims to solve the problem of how to detect the working state of the projector in the structured light module, particularly whether the projector's working environment is in a strong light environment. Figure 1 As shown, the projector state detection method for the structured light module provided in this embodiment includes the following steps.

[0023] Step 101: Use the infrared camera of the structured light module to simultaneously acquire infrared images and speckle images, and determine the face frame and key points of the human eyeballs in the speckle image based on the infrared image. In the infrared image, the human eyes are not closed and are not blocked by opaque objects.

[0024] Specifically, in scenarios where structured light modules are used for facial recognition (e.g., face recognition scenarios), the infrared camera of the structured light module can be used to capture the facial image during the facial recognition process. If the projector is not in operation when capturing the image, the captured facial image will be a general infrared image, denoted as I. face If the projector is active when capturing images, the captured face image will be a speckle pattern, denoted as S. face When capturing images of the same face, the capture I... face and capture S face When the action interval is very short (negligible) or the face does not move during image capture, it can be considered that I face and S face The captured image content is spatially aligned, and the infrared camera simultaneously acquires infrared images and speckle images.

[0025] To ensure the accuracy of the final detection results, the infrared and speckle images acquired synchronously in the early stages must meet certain requirements. Specifically, the infrared image must show the human eye not closed and not obstructed by any opaque object. The opaque object can be any object that obstructs the view of the human eye but prevents the image of the eye from being displayed in the image. In other words, the infrared image must show the human eye open, and the area around the eye can be either unobstructed or obstructed by a transparent object, such as eyeglasses.

[0026] For the synchronously acquired infrared and speckle images, since they are spatially aligned, identical image content will also be located in the same position within the images. Therefore, by recognizing faces in the infrared image and determining the positions of the face bounding boxes and key points of the eyes, the corresponding face bounding boxes and key points of the eyes can be determined in the synchronously acquired speckle image based on the spatial alignment. Since subsequent image detection processing primarily targets the face bounding boxes, the image quality of these face bounding boxes should meet certain basic requirements during the initial image acquisition, such as resolution, contrast, and pixel area.

[0027] In some embodiments, determining the face frame and key points of the human eyeball in the speckle image based on the infrared image may include the following steps a to b.

[0028] Step a: Use a pre-trained first classifier to identify the acquired infrared images and retain those classified as having open eyes and not being blocked by opaque objects.

[0029] The first classifier's function is to identify and classify the input infrared image, and the categories it can identify must include at least one category where the human eye is not closed and is not obstructed by an opaque object. Of course, other categories that the first classifier can identify are not limited in this embodiment; for example, it can also include categories such as "the human eye is closed and not obstructed by an opaque object" and "the human eye is obstructed by an opaque object." The first classifier is a pre-trained model, and this embodiment does not limit the pre-training process of the first classifier.

[0030] Specifically, when identifying infrared images, a pre-trained first classifier can be used to identify the collected infrared images, so as to identify and retain infrared images that are classified as having open eyes and not being blocked by opaque objects from among many infrared images.

[0031] Step b: Based on the retained infrared image, determine the face bounding box and key points of the human eyeball in the speckle image acquired synchronously with it.

[0032] Specifically, for the infrared image identified by the first classifier as having eyes that are not closed and not obscured by opaque objects, the positions of the face bounding box and key points of the human eyeball can be determined through target image detection. Then, based on the image spatial alignment relationship, the infrared image can be reconstructed. Figure 1 The positions of the face frame and key points of the human eyeball in the speckle image collected simultaneously.

[0033] Step 102: If speckle abnormalities are detected within the face frame, identify whether there is high-intensity reflected light at key points of the human eyeball.

[0034] Specifically, after determining the face bounding box by target detection in the infrared image, and determining the face bounding box in the speckle image based on the spatial alignment relationship, target detection technology can be used to detect whether speckle anomalies occur within the face bounding box in the speckle image. Specific manifestations of speckle anomalies include, but are not limited to, missing speckles or abnormal speckle quality (blurring, tomography), etc.

[0035] In some embodiments, detecting whether there is speckle abnormality within the face frame may include the following steps c to d.

[0036] Step c: Use a pre-trained second classifier to identify speckles within the face bounding box to determine whether there are speckle anomalies within the face bounding box.

[0037] The second classifier identifies the location of face bounding boxes in the input speckle map to classify the speckle map. Each class it can identify must include at least one category where speckle anomalies exist within the face bounding box. However, this embodiment does not limit the other categories that the second classifier can identify; for example, it may also include categories where speckle anomalies do not exist within the face bounding box. The second classifier is a pre-trained model, and this embodiment does not limit the pre-training process of the second classifier.

[0038] Specifically, when identifying speckle patterns, a pre-trained second classifier can be used to identify the collected speckle patterns (face bounding boxes) to identify and retain speckle patterns classified as having speckle anomalies within face bounding boxes from among numerous speckle patterns.

[0039] To improve the classification efficiency of the second classifier, the speckle image can be preprocessed before recognition. For example, the image region of the face bounding box can be extracted from the speckle image to obtain the face bounding box speckle image. Then, a pre-trained second classifier can be used to identify the face bounding box speckle image to determine whether there are speckle anomalies within the face bounding box. Obtaining a face bounding box speckle image with less image content by image cropping and inputting it into the second classifier for processing can reduce the amount of data processed by the second classifier, thereby shortening the processing time for identifying speckle anomalies and improving processing efficiency.

[0040] Step d: If no speckle abnormalities are detected within the face frame, the projector is determined to be in normal condition.

[0041] If no abnormalities are detected in the speckle pattern within the face frame during speckle pattern detection, it can be assumed that the speckle projected by the projector is normal, the projector is not obstructed, and the projector current parameters are normal, meaning the projector is in a normal state.

[0042] If an abnormal speckle pattern is detected within the face frame during speckle pattern detection, the cause of the abnormality needs to be further determined. This embodiment primarily addresses the issue of the projector being in a strong light environment as the cause of the speckle abnormality, and the method employed is to identify whether high-intensity reflected light exists at key points on the human eyeball.

[0043] In actual use of structured light modules for facial recognition, the expected facial position is that the face is facing the speckle projector, the projector is working, and the emitted light is projected onto the face. When the facial recognition action is performed, if the user's eyes are not obstructed by an opaque object and their eyes are open, the light emitted by the projector travels through the air medium to the eyeball and is reflected. A portion of the reflected light travels through the air medium to the infrared camera, resulting in a high-intensity reflection at the eye position in the corresponding speckle pattern (e.g., ...). Figure 2 (As shown).

[0044] Based on this, in the case of detecting speckle abnormalities within the face frame of the speckle pattern, this embodiment can further determine whether there is high-intensity reflection at the human eyeball through image recognition, and then determine whether the speckle abnormality is caused by the projector being in a strong light environment.

[0045] In some embodiments, identifying whether there is high-intensity reflected light at key points of the human eyeball may include: using a pre-trained third classifier to identify the image at the key points of the human eyeball to determine whether there is high-intensity reflected light at the key points of the human eyeball.

[0046] The third classifier's role is to identify and classify the input speckle pattern, ensuring that at least one category is identified where high-intensity reflected light exists at key points on the human eye. However, this embodiment does not limit the categories that the third classifier can identify; for example, it could also include categories where high-intensity reflected light does not exist at key points on the human eye. The third classifier is a pre-trained model, and this embodiment does not limit the pre-training process of the third classifier.

[0047] Specifically, when identifying speckle patterns, a pre-trained third classifier can be used to identify the key points of the human eye in speckle patterns with abnormal speckle patterns within the collected face frame, so as to identify whether there is high-intensity reflected light in the images at the key points of the human eye.

[0048] Step 103: If present, determine that the projector is in a strong light environment.

[0049] After the identification and judgment in step 102, if the image at the key point of the human eyeball in the currently identified speckle pattern (speculiar anomaly exists within the face frame) contains high-intensity reflected light, it is considered that the speckle anomaly may be caused by strong ambient light. The energy of the speckle projected by the rated power of the projector is submerged in the ambient light, thus preventing the speckle from being identified in the speckle pattern (or making the speckle abnormality apparent). Therefore, in this embodiment, when a speckle anomaly appears within the face frame in the speckle pattern, and high-intensity reflected light is identified at the key point of the human eyeball, it is considered that the projector is currently in a strong light environment.

[0050] Furthermore, after step 102, specifically after identifying whether there is high-intensity reflected light at the key point of the human eyeball, it may also include: if there is high-intensity reflected light at the key point of the human eyeball in only one of the two human eyes, then it is determined that the projector is in an occluded state.

[0051] In this embodiment, the speckle pattern may be abnormal when the projector is in a strong light environment, but this does not exclude the possibility that the abnormal speckle pattern may be caused by other situations, such as the projector being blocked.

[0052] When the cause of speckle anomaly is solely due to the projector being in a strong light environment, then high-intensity reflected light should be present at the key points of the eyeballs corresponding to both eyes in the speckle diagram, i.e., as shown below. Figure 2 As shown in the diagram. However, if only one eye of a person has a high-intensity reflected light at its key point, while the other eye does not, it can be assumed that the light path between the projector and that eye is blocked, i.e., the projector is blocked, resulting in a situation where only one eye of a person has a high-intensity reflected light at its key point.

[0053] Based on this, after executing step 102, if it is detected that only one of the two eyes in the speckle pattern has high-intensity reflected light at the key point of the eyeball, then it is determined that the projector is in a blocked state.

[0054] The above method determines the projector's state based on whether there is high-intensity reflected light at key points of the human eyeball, such as whether the projector is in a strong light environment or whether it is obstructed. The following embodiments, building upon the aforementioned embodiments, will illustrate how to detect the projector's state when the human eye is not closed and is obstructed by a transparent object (here, both eyes are obstructed by a transparent object; a typical scenario is when the human eye is wearing glasses, and the transparent object is the lens).

[0055] In execution Figure 1 During the process of the method steps shown, such as Figure 3 As shown, steps 104 to 106 can also be performed.

[0056] Step 104: Detect whether the human eye in the infrared image is blocked by a light-transmitting object. If so, determine the corresponding area of ​​the human eye blocked by the light-transmitting object in the speckle image based on the infrared image.

[0057] Following step 101, based on the synchronously acquired infrared and speckle images, target detection technology can be used to detect whether the human eye in the infrared image is obstructed by a translucent object. The characteristic of an eye being obstructed by a translucent object is that the obstruction can be detected not only at the location of the eye but also within the area obstructed by the object.

[0058] In some embodiments, a pre-trained first classifier may be used to identify the acquired infrared images and retain those classified as infrared images whose human eyes are blocked by light-transmitting objects.

[0059] The first classifier, as described above, can identify categories including, in addition to those where the eyes are not closed and not obstructed by opaque objects, categories where the eyes are obstructed by translucent objects. Of course, this embodiment does not limit the other categories that the first classifier can identify; for example, it may also include categories such as the eyes not being obstructed by translucent objects.

[0060] It should be noted that the infrared image detected in this embodiment still needs to meet the requirement that the human eye is not closed and is not blocked by an opaque object. However, based on this requirement, the categories for identification are further refined, namely, the category of human eye being blocked by a transparent object (the corresponding category of human eye not being blocked by a transparent object).

[0061] Specifically, when identifying infrared images, a pre-trained first classifier can be used to identify the collected infrared images, so as to further identify and retain infrared images classified as having eyes blocked by light-transmitting objects from among the many infrared images in which the human eyes are not closed and are not blocked by opaque objects.

[0062] After obtaining an infrared image of a person's eyes that are not closed and are blocked by a light-transmitting object, the light-transmitting object is detected to determine its position. Then, based on the spatial alignment relationship between the synchronously acquired infrared image and the speckle image, the occlusion area of ​​the human eye in the speckle image is determined according to the occlusion area of ​​the light-transmitting object in the infrared image.

[0063] Step 105: If speckle abnormalities are detected within the face frame, identify whether there is high-intensity reflected light in the occluded area.

[0064] For details on how to detect the presence of speckle anomalies within the face frame, please refer to the corresponding content in step 102. Unlike step 102, this step, when speckle anomalies are detected within the face frame, identifies whether high-intensity reflected light exists within the occluded area corresponding to the transparent object obscuring the viewer's eye in the speckle map. The specific method for identifying high-intensity reflected light is similar to the method used in step 102 for identifying the presence of high-intensity reflected light at key points on the viewer's eyeball.

[0065] Specifically, if no abnormalities are detected in the speckle pattern within the face frame during speckle pattern detection, it can be assumed that the speckle projected by the projector is normal, the projector is not obstructed, and the projector current parameters are normal, meaning the projector is in a normal state.

[0066] If an abnormal speckle pattern is detected within a face frame during speckle pattern detection, the cause of the abnormality needs to be further determined. This embodiment primarily addresses the issue of speckle anomalies caused by the projector being in a strong light environment. The method employed is to identify whether high-intensity reflected light exists within the obstructed area of ​​a translucent object.

[0067] In actual use of structured light modules for facial recognition, the expected facial orientation is that the face is facing the speckle projector, the projector is operational, and the emitted light is projected onto the face. When the facial recognition process is underway, if the user's eyes are obstructed by a translucent object (such as eyeglass lenses) and their eyes are open, the light emitted by the projector, after passing through the air and the translucent object, is reflected upon reaching the eye. A portion of the reflected light passes through the translucent object and air to reach the infrared camera, resulting in a high-intensity reflection at the eye location in the corresponding speckle pattern. Simultaneously, the light emitted by the projector, after passing through the air and reaching the translucent object, is reflected again, with a portion of the reflected light passing through the air to reach the infrared camera. This results in a high-intensity reflection within the obstructed area of ​​the translucent object in the corresponding speckle pattern (e.g.,...). Figure 4 (As shown).

[0068] Based on this, in the case of detecting speckle abnormalities within the face frame of the speckle pattern, this embodiment can further determine whether there is high-intensity reflection in the area blocked by the light-transmitting object at the human eye through image recognition, and then determine whether the cause of the speckle abnormality is that the projector is in a strong light environment.

[0069] In some embodiments, identifying whether high-intensity reflected light exists within the occluded area may include: using a pre-trained third classifier to identify the image of the occluded area to determine whether high-intensity reflected light exists within the occluded area.

[0070] The third classifier, as described above, can identify categories including, in addition to those related to high-intensity reflected light at key points of the human eyeball, high-intensity reflected light within the obstructed area where the eye is blocked by a translucent object. Of course, this embodiment does not limit other categories that the third classifier can identify; for example, it may also include categories such as the absence of high-intensity reflected light within the obstructed area where the eye is blocked by a translucent object.

[0071] Specifically, when identifying speckle patterns, a pre-trained third classifier can be used to identify the occluded area where the human eye is blocked by a transparent object in the collected speckle pattern with abnormal speckle patterns within the face frame, so as to identify whether there is high-intensity reflected light in the image within the occluded area.

[0072] Step 106: If present, determine that the projector is in a strong light environment.

[0073] After the identification and judgment in step 105, if the image in the currently identified speckle pattern (where speckle anomalies exist within the face frame) shows high-intensity reflected light in the area obscured by a transparent object at the viewer's eye, then the speckle anomaly is considered to be caused by strong ambient light. The energy of the speckle projected by the projector's rated power is submerged in the ambient light, thus preventing the speckle pattern from being identified (or indicating a speckle anomaly). Therefore, in this embodiment, when speckle anomalies appear within the face frame in the speckle pattern, and high-intensity reflected light is identified in the image within the area obscured by a transparent object at the viewer's eye, it is considered that the projector is currently in a strong light environment.

[0074] Furthermore, after step 105, specifically after identifying the presence of high-intensity reflected light within the occluded area, the method may further include: if high-intensity reflected light exists in only one of the two occluded areas corresponding to the two human eyes, then it is determined that the projector is in an occluded state.

[0075] In this embodiment, the speckle pattern may be abnormal when the projector is in a strong light environment, but this does not exclude the possibility that the abnormal speckle pattern may be caused by other situations, such as the projector being blocked.

[0076] When the speckle anomaly is solely caused by the projector being in a bright light environment, then high-intensity reflected light should exist in the areas of obstruction corresponding to both eyes in the speckle diagram, i.e., as shown below. Figure 4 As shown in the diagram. However, if only one eye of a person has a high-intensity reflected light in its occluded area, while the other eye does not, it can be assumed that the light path between the projector and the transparent object corresponding to that eye is blocked, i.e., the projector is blocked. This results in a situation where only one eye of a person has a high-intensity reflected light in its occluded area.

[0077] Based on this, after executing step 105, if it is detected that only one of the two occluded areas corresponding to the two human eyes in the speckle pattern has high-intensity reflected light, then it is determined that the projector is in an occluded state.

[0078] In addition, the following steps are included in the process of executing the method described in this embodiment: if an abnormal speckle pattern is detected within the face frame, and it is identified that there is no high-intensity reflected light at the key points of the human eyeball and in the occluded area, then it is determined that the projector is in an occluded state.

[0079] This step can serve as a fallback reason in the above-mentioned determination of abnormal speckle patterns within the face frame. That is, if the projector's state cannot be determined through the above methods and steps, it can be basically assumed that the projector is in an occluded state.

[0080] Compared with related technologies, the embodiments of this application mainly solve the problem of effectively acquiring the working status of the projector in facial recognition scenarios when the ambient light is strong. This approach is low-cost and does not require an additional distance sensor. When the ambient light is strong, the energy of the speckle projected by the projector's rated power is submerged in the ambient light, making the speckle undetectable in the speckle image. However, the structured light module cannot determine whether the projector is blocked, not lit, or due to excessive ambient light. If the projector is blocked or not lit, the user needs to be prompted to unblock it, and the structured light module needs to disable the structured light facial recognition function. If the structured light module rules out that the projector is blocked or not lit, the method steps shown in this embodiment can be executed to determine whether the abnormal speckle is caused by excessive ambient light.

[0081] Of course, in practical applications, other methods can also be used to try to complete the face recognition function, such as increasing the power of the projector, requiring the person being recognized to move closer, or switching to other identity verification modes due to the lack of structured light face recognition environment conditions, etc., depending on the specific business logic of the product itself.

[0082] Another embodiment of the present invention relates to an electronic device, such as Figure 5 As shown, it includes at least one processor 302; and a memory 301 communicatively connected to at least one processor 302; wherein the memory 301 stores instructions executable by at least one processor 302, the instructions being executed by at least one processor 302 to enable at least one processor 302 to execute any of the above method embodiments.

[0083] The memory 301 and processor 302 are connected via a bus, which may include any number of interconnecting buses and bridges. The bus connects various circuits of one or more processors 302 and memory 301 together. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 302 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 302.

[0084] Processor 302 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 301 can be used to store data used by processor 302 during operation.

[0085] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements any of the above-described method embodiments.

[0086] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0087] Those skilled in the art will understand that the above embodiments are specific examples of implementing the present invention, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of the present invention.

Claims

1. A method for detecting the projector state of a structured light module, characterized in that, include: Infrared images and speckle images are simultaneously acquired using the infrared camera of the structured light module; The pre-trained first classifier is used to identify the acquired infrared images and retain those classified as having open eyes and not being blocked by opaque objects; based on the retained infrared images, the face bounding box and key points of the human eyeballs in the speckle image acquired at the same time are determined. If an abnormal speckle pattern is detected within the face frame, identify whether there is high-intensity reflected light at key points of the human eyeball; if so, determine that the projector is in a strong light environment. Detect whether the human eye in the infrared image is blocked by a light-transmitting object. If so, determine the corresponding area of ​​the human eye blocked by the light-transmitting object in the speckle image based on the infrared image. If abnormal speckle pattern is detected within the face frame, identify whether high-intensity reflected light exists within the occluded area; If present, it is determined that the projector is in a strong light environment; If an abnormal speckle pattern is detected within the face frame, and it is identified that there is no high-intensity reflected light at the key points of the human eyeball and within the occluded area, then it is determined that the projector is in an occluded state.

2. The method according to claim 1, characterized in that, Detecting whether there are speckle abnormalities within the face frame includes: A pre-trained second classifier is used to identify speckle patterns within the face bounding box to determine whether there are any speckle anomalies within the face bounding box. If no speckle abnormalities are detected within the face frame, the projector is determined to be in normal condition.

3. The method according to claim 1, characterized in that, Identifying whether high-intensity reflected light exists at key points of the human eyeball includes: A pre-trained third classifier is used to identify the key points of the human eyeball in the image to determine whether there is high-intensity reflected light at the key points of the human eyeball.

4. The method according to claim 1, characterized in that, After identifying whether there is high-intensity reflected light at key points of the human eyeball, the method further includes: If, in the eyes of two people, only one person's eyeball shows a high intensity of reflected light at a key point, then the projector is determined to be in a blocked state.

5. The method according to claim 1, characterized in that, After identifying whether high-intensity reflected light exists within the obstructed area, the method further includes: If high-intensity reflected light exists in only one of the two occluded areas corresponding to the two human eyes, then the projector is determined to be in an occluded state.

6. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the projector state detection method of the structured light module as described in any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the projector state detection method of the structured light module as described in any one of claims 1 to 5.

Images