Projector state detection method of structured light module, electronic equipment and storage medium
Through infrared cameras, infrared images and speckle images are collected simultaneously, key points of the face frame and eyeball, and the projector status is determined, which solves the problem that it is difficult to obtain the projector's working state when the ambient light is strong, and achieves a low-cost and efficient state Test.
Patent Information
- Application Number
- CN202411048092.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-01
AI Technical Summary
In the face-scan application of structured light modules, it is difficult to effectively obtain the working state of the projector when the ambient light is strong, and the prior art requires additional low-precision distance sensors, which increases cost and volume.
The infrared camera of the structured light module synchronously collects infrared and speckle images. By identifying the face frame and the key points of the human eyeball, we can determine whether the projector is in a strong light environment, and identifying whether there is high-intensity reflected light at the key points of the human eyeball. Determine the projector status.
Without the need for additional distance sensors, it is effective to determine whether the projector is blocked or in a strong light environment, reducing cost and volume and improving detection efficiency.
Smart Images

Figure CN119027947A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of machine vision, and in particular to a projector state detection method of a structured light module, an electronic device and a storage medium. Background Art
[0002] The structured light module has high requirements for efficiency and security in face recognition applications. The module obtains information about the working status of the structured light projector (referred to as "projector"), such as whether the projector is blocked and whether the speckle pattern currently projected by the projector is normal, which plays an important role in the subsequent logic of the face recognition application.
[0003] The existing technology adds a low-precision distance sensor near the projector for distance measurement, and infers whether the projector is blocked by the distance sensor's feedback of the closer distance value. In addition, when an object moves in front of the structured light module, the distance sensor senses that there is an object moving in the environment in front, and can infer a suitable working state of the projector based on the perceived rough distance information, and verify whether the current working parameters of the projector are suitable with the high-precision three-dimensional information obtained when the structured light module is working.
[0004] However, when the ambient light is strong, the energy of the speckle projected by the rated power of the projector is submerged in the ambient light, and the speckle cannot be identified in the image or the speckle quality is low. However, the module cannot confirm whether the projector is blocked, not bright, or because the ambient light is too strong. In addition, according to the current market feedback on face recognition modules, there is a strong demand for reducing costs and volume. Some module products do not have extra cost budget and location to add a low-precision distance sensor, which also puts higher requirements on detecting the working status of the projector. Summary of the invention
[0005] The purpose of the embodiments of the present invention is to provide a projector status detection method, electronic device and storage medium of a structured light module, which can effectively solve the problem of effectively obtaining the working status of the projector when the ambient light is strong in the face recognition scenario using the structured light module, and has low cost and does not require the additional setting of a distance sensor.
[0006] In order to solve the above technical problems, an embodiment of the present invention provides a method for detecting the state of a projector of a structured light module, comprising:
[0007] Using an infrared camera of a structured light module to synchronously collect an infrared image and a speckle image, and determining a face frame and key points of a human eye in the speckle image based on the infrared image, wherein the human eye in the infrared image is not closed and is not blocked by an opaque object;
[0008] When a speckle anomaly is detected in the face frame, identifying whether high-intensity reflected light exists at key points of the eyeball of the person;
[0009] If present, it is determined that the projector is in a strong light environment.
[0010] An embodiment of the present invention further provides an electronic device, including:
[0011] at least one processor; and,
[0012] a memory communicatively connected to the at least one processor; wherein,
[0013] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the projector state detection method of the structured light module as described above.
[0014] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the projector state detection method of the structured light module as described above.
[0015] Compared with the prior art, the embodiments of the present invention use the infrared camera of the structured light module to synchronously collect infrared images and speckle images, and determine the face frame and key points of the human eyeball in the speckle image based on the infrared image, wherein the human eye in the infrared image is not closed and is not blocked by opaque objects; when the presence of speckle anomalies in the face frame is detected, whether there is high-intensity reflected light at the key points of the human eyeball is identified; if there is, it is determined that the projector is in a strong light environment. This solution uses the projector to work in a strong light environment. Since the eyeball of the collected face is equivalent to a convex lens that has a reflective effect on light, high-intensity reflections will appear at the eyeball position in the speckle image. Therefore, when the presence of speckle anomalies in the face frame is detected, it can be determined whether the speckle anomaly in the speckle image is caused by the projector being in a strong light environment by identifying whether there is high-intensity reflected light at the key points of the human eyeball in the speckle image. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 The specific process of the projector state detection method of the structured light module according to the embodiment of the present invention is Figure 1 ;
[0017] Figure 2 is a schematic diagram of a scene of high-intensity reflected light at key points of a human eyeball according to an embodiment of the present invention;
[0018] Figure 3 The specific process of the projector state detection method of the structured light module according to the embodiment of the present invention is Figure 2 ;
[0019] Figure 4 is a schematic diagram of a scene of high-intensity reflected light in an area blocked by a light-transmitting object at a human eye according to an embodiment of the present invention;
[0020] Figure 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0021] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the following will be described in detail with reference to the accompanying drawings. However, it will be appreciated by those skilled in the art that in the various embodiments of the present invention, many technical details are provided in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can be implemented.
[0022] One embodiment of the present invention relates to a method for detecting the state of a projector of a structured light module. The execution subject of the method can be the structured light module itself or an electronic device with image processing function connected to the structured light module. The method aims to solve the problem of how to detect the working state of the projector in the structured light module, especially to detect whether the working environment of the projector is in a strong light environment. Figure 1 As shown, the projector state detection method of the structured light module provided in this embodiment includes the following steps.
[0023] Step 101: using the infrared camera of the structured light module to synchronously collect the infrared image and the speckle image, and determining the face frame and the key points of the human eye in the speckle image based on the infrared image, wherein the human eyes in the infrared image are not closed and are not blocked by opaque objects.
[0024] Specifically, in a scenario where the structured light module is used for face scanning (such as a face recognition scenario), the infrared camera of the structured light module can be used to capture the face image when the face is scanned. If the projector is not in a working state when capturing the image, the captured face image is a general infrared image, which is recorded as I face ; If the projector is in working state when capturing the image, the captured face image is a speckle pattern, denoted as S face When capturing images of the same face, capture I face and capture S face When the interval between actions is very short (the interval can be ignored) or the face does not move during the image capture, it can be considered that I face and S face The captured image content is spatially aligned, and the infrared camera collects the infrared image and speckle pattern synchronously.
[0025] In order to ensure the accuracy of the final detection result in this embodiment, the infrared image and speckle image collected synchronously in the early stage must meet certain requirements, that is, the human eye is not closed and is not blocked by an opaque object in the infrared image. Among them, the opaque object can be a light-blocking object that blocks the position of the human eye and prevents the image of the eye from being displayed in the image. In other words, the infrared image must reflect the open state of the human eye, and there can be no blocker at the human eye, or there can be a light-transmitting blocker, such as a transparent material blocker such as glasses.
[0026] For the infrared image and speckle image collected synchronously, since the two are aligned in space, the position of the same image content in the image is also the same. Therefore, by identifying the face in the infrared image and determining the position of the face frame and the key points of the eye, the corresponding face frame and the key points of the eye can be determined in the speckle image collected synchronously based on the spatial alignment relationship. Since the subsequent image detection processing is mainly aimed at the face frame, the image quality of the face frame should also meet certain basic requirements when collecting images in the early stage, such as resolution, contrast, pixel area, etc.
[0027] In some embodiments, determining the face frame and key points of the human eye in the speckle image based on the infrared image may include the following steps a to b.
[0028] Step a: Use the pre-trained first classifier to identify the collected infrared images, and retain the infrared images classified as those in which the human eyes are not closed and are not blocked by opaque objects.
[0029] Among them, the function of the first classifier is to recognize the input infrared image to classify the infrared image, and the categories that can be recognized at least include the human eyes are not closed and are not blocked by opaque objects. Of course, other categories that can be recognized by the first classifier are not limited in this embodiment, for example, it can also include the human eyes are closed and not blocked by opaque objects, the human eyes are blocked by opaque objects, etc. The first classifier is a pre-trained model, and this embodiment does not limit the pre-training process of the first classifier.
[0030] Specifically, when recognizing infrared images, a pre-trained first classifier may be used to recognize the collected infrared images, so as to recognize and retain infrared images classified as those in which the human eyes are not closed and are not blocked by opaque objects from among numerous infrared images.
[0031] Step b: Based on the retained infrared image, determine the face frame and key points of the human eye in the speckle image collected synchronously with the retained infrared image.
[0032] Specifically, for the infrared image classified by the first classifier as a human eye that is not closed and not blocked by an opaque object, the position of the face frame and the key points of the human eyeball can be determined through target image detection. Then, based on the image spatial alignment relationship, the image with the infrared Figure 1 It synchronizes the positions of the key points of the face frame and the eyeballs in the collected speckle image.
[0033] Step 102: When a speckle anomaly is detected in the face frame, identify whether there is high-intensity reflected light at key points of the eyeball.
[0034] Specifically, after determining the face frame by performing target detection on the infrared image, and determining the face frame in the speckle image according to the face frame in the infrared image through the spatial alignment relationship, the target detection technology can be used to detect whether the speckle pattern in the face frame in the speckle image has speckle abnormality. The specific manifestations of speckle abnormality include but are not limited to speckle missing or speckle quality abnormality (blurring, fault), etc.
[0035] In some embodiments, detecting whether there is a speckle abnormality in the face frame may include the following steps c to d.
[0036] Step c: Use the pre-trained second classifier to identify the speckle in the face frame to determine whether there is a speckle abnormality in the face frame.
[0037] The second classifier is used to identify the position of the face frame in the input speckle image to classify the speckle image, and the identifiable categories at least include the presence of speckle anomaly in the face frame. Of course, other categories that can be identified by the second classifier are not limited in this embodiment, for example, they can also include categories such as the absence of speckle anomaly in the face frame. The second classifier is a pre-trained model, and this embodiment does not limit the pre-training process of the second classifier.
[0038] Specifically, when the speckle pattern is recognized, a pre-trained second classifier may be used to recognize the collected speckle pattern (face frame), so as to recognize and retain the speckle pattern classified as having speckle abnormality in the face frame from among numerous speckle patterns.
[0039] In order to improve the recognition and classification efficiency of the second classifier, the speckle pattern can be preprocessed before recognition. For example, the image area of the face frame can be cut out from the speckle pattern to obtain the face frame speckle pattern, and then the pre-trained second classifier can be used to recognize the face frame speckle pattern to determine whether there is a speckle anomaly in the face frame. The face frame speckle pattern with less image content is obtained by image interception and input into the second classifier for processing, which can reduce the amount of data processed by the second classifier, thereby shortening the processing time for identifying speckle anomalies and improving processing efficiency.
[0040] Step d: When it is detected that there is no speckle anomaly in the face frame, it is determined that the projector is in a normal state.
[0041] If no abnormality is detected in the speckle pattern within the face frame, it can be considered that the speckle projected by the projector is normal, the projector is not blocked, and the projector current parameters are normal, that is, the projector is in a normal state.
[0042] If abnormal speckle is detected in the face frame during the detection of the speckle pattern, it is necessary to further determine the cause of the abnormal speckle. In this embodiment, the cause of the abnormal speckle is mainly detected when the projector is in a strong light environment, and the method used is to identify whether there is high-intensity reflected light at the key points of the human eyeball.
[0043] When the structured light module is actually used for face scanning, the expected face state is that the face is facing the speckle projector, the projector is working, and the emitted light is projected onto the face. When the face scanning action is in progress, if the face scanning person's eyes are not blocked by opaque objects and the eyes are open, the light emitted by the projector passes through the air medium and then reaches the eyeball and is reflected. A part of the reflected light passes through the air medium to the infrared camera, and then in the corresponding speckle pattern, the position of the human eye is manifested as a high-intensity reflection at the human eyeball (such as Figure 2 shown).
[0044] Based on this, when the present embodiment detects the presence of speckle anomaly in the face frame of the speckle pattern, it can further determine whether there is high-intensity reflection at the eyeball through image recognition, and then judge whether the cause of the speckle anomaly is caused by the projector being in a strong light environment.
[0045] In some embodiments, identifying whether there is high-intensity reflected light at key points of a person's eye may include: using a pre-trained third classifier to identify images at key points of a person's eye to determine whether there is high-intensity reflected light at key points of a person's eye.
[0046] The function of the third classifier is to identify the input speckle pattern to classify the speckle pattern, and the categories that can be identified at least include the existence of high-intensity reflected light at the key points of the human eyeball. Of course, other categories that can be identified by the third classifier are not limited in this embodiment, for example, they can also include categories such as the absence of high-intensity reflected light at the key points of the human eyeball. The third classifier is a pre-trained model, and this embodiment does not limit the pre-training process of the third classifier.
[0047] Specifically, when the speckle pattern is recognized, a pre-trained third classifier may be used to recognize the image at the key point of the human eye in the acquired speckle pattern with speckle abnormality in the face frame, so as to identify whether the image at the key point of the human eye has high-intensity reflected light.
[0048] Step 103: If so, determine that the projector is in a strong light environment.
[0049] After the identification and judgment in step 102, if there is high-intensity reflected light in the image at the key point of the human eyeball in the currently identified speckle pattern (there is a speckle anomaly in the face frame), it is considered that the cause of the speckle anomaly may be that the current ambient light is strong, and the energy of the speckle projected by the rated power of the projector is submerged in the ambient light, so that the speckle (or speckle anomaly) cannot be identified in the speckle pattern. Therefore, in this embodiment, when a speckle anomaly appears in the face frame in the speckle pattern, and it is identified that there is high-intensity reflected light in the image at the key point of the human eyeball, it is considered that the current projector is in a strong light environment.
[0050] In addition, after step 102, specifically after identifying whether there is high-intensity reflected light at the key points of the human eyeball, it may also include: if there is high-intensity reflected light at the key points of the human eyeball of only one of the two human eyes, it is determined that the projector is in a blocked state.
[0051] In this embodiment, when the projector is in a strong light environment, the speckle pattern may be abnormal, but this does not exclude that the cause of the abnormal speckle pattern may also include other situations, such as the projector being blocked.
[0052] When the reason for the speckle anomaly is that the transmitter is in a strong light environment, there should be high-intensity reflected light at the key points of the eyeballs corresponding to the two eyes in the speckle pattern, that is, Figure 2 However, if there is high-intensity reflected light at the key point of the eyeball of only one of the two human eyes, and there is no high-intensity reflected light at the key point of the eyeball of the other human eye, it can be considered that the light path from the light projected by the projector to the human eye on that side is blocked, that is, the projector is blocked, so that there is high-intensity reflected light at the key point of the eyeball of only one of the two human eyes.
[0053] Based on this, after executing step 102, in this embodiment, if it is detected that only one of the two human eyes in the speckle pattern has high-intensity reflected light at the key point of the human eyeball, it is determined that the projector is in a blocked state.
[0054] The above is based on judging whether there is high-intensity reflected light at the key points of the human eyeball to judge the state of the projector, such as whether the projector is in a strong light environment or whether the projector is blocked. The following embodiments will, based on the previous embodiments, explain how to detect the state of the projector when the human eye is not closed and is blocked by a transparent object (here both eyes are blocked by transparent objects, a typical scenario is when the human eye wears glasses and the transparent object is a lens).
[0055] In execution Figure 1 During the steps of the method shown, Figure 3 As shown, the following steps 104 to 106 may also be performed.
[0056] Step 104: Detect whether the human eye in the infrared image is blocked by the transparent object. If so, determine the corresponding blocked area of the human eye in the speckle image based on the infrared image.
[0057] After step 101, for the synchronously collected infrared image and speckle image, it is also possible to detect whether the human eye in the infrared image is blocked by a transparent object by using a target detection technology. The characteristic of the human eye being blocked by a transparent object is that the blocking object can be detected at the human eye in the image, and the human eye can also be detected in the blocked area of the blocking object.
[0058] In some embodiments, a pre-trained first classifier may be used to identify the collected infrared images, and retain the infrared images classified as those in which the human eye is blocked by a light-transmitting object.
[0059] Among them, the first classifier is the same as above, and the categories that it can identify include the human eyes being blocked by a translucent object in addition to the above-mentioned categories that the human eyes are not closed and not blocked by an opaque object. Of course, other categories that the first classifier can identify are not limited in this embodiment, for example, it can also include categories such as the human eyes are not blocked by a translucent object.
[0060] It should be noted that the infrared image detected in this embodiment still needs to meet the requirement that the human eyes are not closed and are not blocked by opaque objects in the infrared image. It is just that on the basis of this requirement, the identified category is further refined, that is, the refined category is the human eyes blocked by translucent objects (the relative refined category is the human eyes not blocked by translucent objects).
[0061] Specifically, when recognizing infrared images, a pre-trained first classifier can be used to recognize the collected infrared images, so as to further identify and retain infrared images classified as those in which the human eyes are blocked by translucent objects from among the numerous infrared images in which the human eyes are not closed and are not blocked by opaque objects.
[0062] After obtaining the infrared image of the human eye that is not closed and is blocked by the transparent object, the transparent object is detected to determine the position of the transparent object, and then according to the spatial alignment relationship between the synchronously collected infrared image and the speckle pattern, the corresponding blocked area of the human eye blocked by the transparent object in the speckle pattern is determined based on the blocked area of the transparent object in the infrared image.
[0063] Step 105: When a speckle anomaly is detected in the face frame, identify whether there is high-intensity reflected light in the occluded area.
[0064] For the process of how to detect whether there is a speckle anomaly in the face frame, refer to the corresponding content in step 102. Different from step 102, this step is to identify whether there is high-intensity reflected light in the occlusion area corresponding to the transparent object that blocks the human eye in the speckle pattern when the speckle anomaly is detected in the face frame. The specific method of identifying high-intensity reflected light is similar to the method of identifying whether there is high-intensity reflected light at the key points of the human eye in step 102.
[0065] Specifically, if no abnormality is detected in the speckle within the face frame when the speckle pattern is detected, it can be considered that the speckle projected by the projector is normal, the projector is not blocked, and the projector current parameters are normal, that is, the projector is in a normal state.
[0066] If abnormal speckle is detected in the face frame during the detection of the speckle pattern, it is necessary to further determine the cause of the abnormal speckle. In this embodiment, the cause of the abnormal speckle is mainly detected when the projector is in a strong light environment, and the method used is to identify whether there is high-intensity reflected light in the shielding area of the light-transmitting object.
[0067] When the structured light module is actually used for face scanning, the expected state of the face is that the face is facing the speckle projector, the projector is working, and the emitted light is projected onto the face. When the face scanning action is in progress, if the face scanning person's eyes are blocked by a translucent object (such as glasses lenses) and the eyes are open, the light emitted by the projector passes through the air and the translucent object medium to reach the eyeball and then reflects. A part of the reflected light passes through the translucent object and the air medium to reach the infrared camera, then in the corresponding speckle pattern, at the position of the human eye, it appears as a high-intensity reflection at the human eyeball; at the same time, the light emitted by the projector passes through the air medium to reach the translucent object and then reflects. A part of the reflected light passes through the air medium to reach the infrared camera, then in the corresponding speckle pattern, at the position of the obstruction area of the translucent object, it appears as a high-intensity reflection in the obstruction area (such as Figure 4 shown).
[0068] Based on this, when the present embodiment detects the presence of a speckle anomaly in the face frame of the speckle pattern, it is possible to further determine through image recognition whether there is a high-intensity reflection in the occluded area of the translucent object at the human eye, and then determine whether the cause of the speckle anomaly is caused by the projector being in a strong light environment.
[0069] In some embodiments, identifying whether there is high-intensity reflected light in the occluded area may include: using a pre-trained third classifier to identify an image of the occluded area to determine whether there is high-intensity reflected light in the occluded area.
[0070] Among them, the third classifier is the same as above, and the categories that it can identify include the existence of high-intensity reflected light at the key points of the human eyeball, as well as the existence of high-intensity reflected light in the occlusion area where the human eye is blocked by a transparent object. Of course, other categories that the third classifier can identify are not limited in this embodiment, for example, it can also include the absence of high-intensity reflected light in the occlusion area where the human eye is blocked by a transparent object.
[0071] Specifically, when recognizing the speckle pattern, a pre-trained third classifier may be used to recognize images of occluded areas where eyes are blocked by transparent objects in the acquired speckle pattern with speckle abnormality within the face frame, so as to identify whether there is high-intensity reflected light in the images in the occluded areas.
[0072] Step 106: If so, determine that the projector is in a strong light environment.
[0073] After the identification and judgment in step 105, if in the currently identified speckle pattern (there is a speckle anomaly in the face frame), there is high-intensity reflected light in the image of the blocked area where the transparent object blocks the human eye, it is considered that the cause of the speckle anomaly may be that the current ambient light is strong, and the energy of the speckle projected by the rated power of the projector is submerged in the ambient light, so that the speckle (or speckle anomaly) cannot be identified in the speckle pattern. Therefore, in this embodiment, when a speckle anomaly appears in the face frame in the speckle pattern, and it is identified that there is high-intensity reflected light in the image of the blocked area where the human eye is blocked by the transparent object, it is considered that the current projector is in a strong light environment.
[0074] In addition, after step 105, specifically after identifying the presence of high-intensity reflected light in the occluded area, it may also include: if there is high-intensity reflected light in only one of the two occluded areas corresponding to the two human eyes, it is determined that the projector is in an occluded state.
[0075] In this embodiment, when the projector is in a strong light environment, the speckle pattern may be abnormal, but this does not exclude that the cause of the abnormal speckle pattern may also include other situations, such as the projector being blocked.
[0076] When the reason for the speckle anomaly is that the transmitter is in a strong light environment, there should be high-intensity reflected light in the occluded areas corresponding to the two human eyes in the speckle pattern, that is, Figure 4 However, if there is high-intensity reflected light in the occlusion area corresponding to only one of the two human eyes, and there is no high-intensity reflected light in the occlusion area corresponding to the other human eye, it can be considered that the light path between the light projected by the projector and the light-transmitting object corresponding to the human eye on that side is blocked, that is, the projector is blocked, so that there is a situation in which high-intensity reflected light is present in the occlusion area corresponding to only one of the two human eyes.
[0077] Based on this, after executing step 105, this embodiment determines that the projector is in an obstructed state if it is detected that there is high-intensity reflected light in only one of the two obstruction areas corresponding to the two human eyes in the speckle pattern.
[0078] In addition, the process of executing the method described in this embodiment also includes the following steps: when a speckle anomaly is detected in the face frame, and it is identified that there is no high-intensity reflected light at the key points of the human eye and in the occluded area, it is determined that the projector is in an occluded state.
[0079] This step can serve as a fallback reason for the above-mentioned determination of the abnormal speckle in the face frame, that is, when the state of the projector cannot be determined by the above-mentioned method steps, it can be basically considered that the projector is in an obstructed state.
[0080] Compared with the related art, the embodiment of the present application mainly solves the problem of effectively obtaining the working status of the projector in the face scanning scene when the ambient light is strong, and it is low-cost and does not require an additional distance sensor. When the ambient light is strong, the energy of the speckle projected by the rated power of the projector is submerged in the ambient light, and the speckle cannot be identified in the speckle image, but the structured light module cannot confirm whether the projector is blocked, not lit, or because the ambient light is too strong. If the projector is blocked or not lit, the user needs to be prompted to release the state of the projector being blocked or not lit, and the structured light module needs to disable the structured light face scanning function. If the structured light module excludes the situation that the projector is blocked or not lit, it is possible to determine whether the cause of the speckle abnormality is due to excessive ambient light by executing the method steps shown in this embodiment.
[0081] Of course, in actual applications, other methods can also be used to try to complete the face scanning function, such as increasing the power of the projector, or requiring the person scanning their face to cooperate and get closer, or switching to other identity verification modes because the environment conditions for structured light face scanning are not met, etc., depending on the specific product's own business logic.
[0082] Another embodiment of the present invention relates to an electronic device, such as Figure 5 As shown, it includes at least one processor 302; and a memory 301 that is communicatively connected to the at least one processor 302; wherein the memory 301 stores instructions that can be executed by the at least one processor 302, and the instructions are executed by the at least one processor 302 so that the at least one processor 302 can execute any of the above method embodiments.
[0083] The memory 301 and the processor 302 are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors 302 and the memory 301 together. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor 302 is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor 302.
[0084] The processor 302 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management and other control functions. The memory 301 can be used to store data used by the processor 302 when performing operations.
[0085] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program, which implements any of the above method embodiments when executed by a processor.
[0086] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0087] Those skilled in the art will appreciate that the above-mentioned embodiments are specific examples for implementing the present invention, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present invention.
Claims
1. A method for detecting the state of a projector of a structured light module, characterized in that: include: Using an infrared camera of a structured light module to synchronously collect an infrared image and a speckle image, and determining a face frame and key points of a human eye in the speckle image based on the infrared image, wherein the human eye in the infrared image is not closed and is not blocked by an opaque object; When a speckle anomaly is detected in the face frame, identifying whether high-intensity reflected light exists at key points of the eyeball of the person; If present, it is determined that the projector is in a strong light environment.
2. The method according to claim 1, characterized in that The determining of the face frame and the key points of the eyeballs in the speckle image based on the infrared image comprises: Using a pre-trained first classifier to identify the collected infrared images, and retaining the infrared images classified as those in which the human eyes are not closed and are not blocked by opaque objects; Based on the retained infrared image, the face frame and key points of the human eyeballs in the speckle image collected synchronously therewith are determined.
3. The method according to claim 1, characterized in that Detecting whether there is a speckle abnormality in the face frame includes: Using a pre-trained second classifier to identify speckles in the face frame to determine whether there is a speckle abnormality in the face frame; When it is detected that there is no speckle anomaly in the face frame, it is determined that the projector is in a normal state.
4. The method according to claim 1, characterized in that: Identifying whether there is high-intensity reflected light at the key points of the human eyeball includes: A pre-trained third classifier is used to identify the image at the key point of the human eyeball to determine whether high-intensity reflected light exists at the key point of the human eyeball.
5. The method according to claim 1, characterized in that After identifying whether there is high-intensity reflected light at the key points of the human eyeball, the method further includes: If there is high-intensity reflected light at the key point of the eyeball of only one of the two human eyes, it is determined that the projector is in an obstructed state.
6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Detecting whether the human eye in the infrared image is blocked by the light-transmitting object, and if so, determining the corresponding blocked area of the human eye in the speckle image blocked by the light-transmitting object based on the infrared image; In the case where a speckle anomaly is detected in the face frame, identifying whether there is high-intensity reflected light in the occluded area; If present, it is determined that the projector is in a strong light environment.
7. The method according to claim 6, characterized in that After identifying whether there is high-intensity reflected light in the shielding area, the method further includes: If high-intensity reflected light exists in only one of the two occlusion areas corresponding to the two human eyes, it is determined that the projector is in an occluded state.
8. The method according to claim 6, characterized in that The method further comprises: When a speckle anomaly is detected in the face frame and it is identified that there is no high-intensity reflected light at the key points of the human eyeball and in the blocked area, it is determined that the projector is in a blocked state.
9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the projector state detection method of the structured light module as described in any one of claims 1 to 8.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the projector state detection method of the structured light module according to any one of claims 1 to 8 is implemented.
Citation Information
Patent Citations
Electronic device control method and apparatus, electronic device, and readable storage medium
CN108683843A
Optical component detection method and device, electronic equipment and readable storage medium
CN108760245A
Face recognition method and device, equipment and storage medium
CN116092158A
Face recognition device and face recognition method
CN116935456A
3D camera system and method for detecting abnormal state of beam projector module for eye protection
US10444331B1
Cited By
Face recognition method and device, equipment and storage medium
CN116092158A
Face recognition method, device and equipment and storage medium
CN116092158B
