Projection display device

Abstract

The embodiment of the invention discloses projection display equipment, which comprises a projection module, a time of flight (TOF) sensor and a main control circuit, and is characterized in that when the main control circuit determines that a gesture recognition function of the projection display equipment is enabled and a target area included in a target depth image collected by the TOF sensor accords with a gesture recognition condition, the gesture recognition function of the projection display equipment is enabled; the number of fingers extending out of the target object is obtained, the projection display device is controlled to execute operation corresponding to the number of the fingers according to a preset target operation mapping relation, and the target area comprises a head area, or the head area and a hand area, or the trunk area and the hand area. According to the embodiment of the invention, under the condition that the main control circuit determines that the target area of the target depth image meets the gesture recognition condition, gesture recognition is carried out on the hand area, and unintentional hand actions of a user are prevented from being collected by the projection display equipment and recognized as control operation on the equipment, so that the accuracy of gesture control is improved, and the user experience is improved. And the user experience of using the projection display equipment is improved.

Description

Technical Field

[0001] This application relates to the field of projection equipment technology, and to, but is not limited to, a projection display device. Background Technology

[0002] With continuous technological advancements, projection display devices have made significant progress in key technologies such as light sources and lenses, leading to a growing number of users choosing projection display devices for audio-visual entertainment. Common projection display devices are typically controlled via remote controls or other remote-controlled terminal devices, such as smartphones.

[0003] In related technologies, to improve the human-computer interaction experience, some projection display devices can recognize user gestures by integrating image sensors, allowing users to control the device in a contactless manner. However, during the use of projection display devices, users may inadvertently move their hands due to communicating with others, picking up objects, or other reasons. These unintentional hand movements may be collected and recognized by the projection display device as control operations, leading the device to execute responses that are not intended by the user, thus affecting the user experience of using the projection display device. Summary of the Invention

[0004] In view of this, embodiments of this application provide a projection display device to solve the technical problem of how to identify which of a user's hand gestures are intended to control the projection display device when recognizing user gestures. The projection display device can more accurately respond to the user's actual needs, reduce accidental device operation due to misrecognition, and thus improve overall user satisfaction and user experience.

[0005] This application provides a projection display device, the projection display device comprising:

[0006] A projection module is used to project an optical signal corresponding to a projection signal and display an image or video corresponding to the projection signal, wherein the projection signal is an image signal or a video signal.

[0007] A time-of-flight (TOF) sensor is used to acquire depth images. The TOF sensor is disposed on the side of the projection module in the projection display device that projects the light signal. The acquisition area of ​​the TOF sensor and the projection area of ​​the projection module overlap.

[0008] The main control circuit is used to acquire the depth image collected by the TOF sensor, perform gesture recognition on the depth image collected by the TOF sensor, and control the projection display device to perform the operation corresponding to the gesture recognition result.

[0009] Once the main control circuit determines that the gesture recognition function of the projection display device is enabled, and the target area included in the target depth image acquired by the TOF sensor meets the gesture recognition conditions, the main control circuit performs gesture recognition on the hand area, obtains the number of outstretched fingers of the target object, and controls the projection display device to execute the operation corresponding to the number of fingers according to a preset target operation mapping relationship.

[0010] The target area includes a head area, or the target area includes the head area and the hand area, or the target area includes the torso area and the hand area; the target operation mapping relationship includes the mapping relationship between a preset number of fingers and a preset operation of the projection display device, and the gesture recognition result includes the number of fingers extended by the object.

[0011] In the above technical solution, when the gesture recognition function of the projection display device is enabled, and the main control circuit determines that the target area included in the target depth image acquired by the TOF sensor meets the gesture recognition conditions, gesture recognition is performed on the hand area. Since the target area can be the head area, or the head area and hand area, or the torso area and hand area, each different target area contains image content outside the hand area, providing data reference for determining whether the gesture recognition conditions are met. When the main control circuit determines that the target area meets the gesture recognition conditions, it then performs gesture recognition on the hand area to improve the reliability of the recognition results and reduce the possibility of the projection display device responding to unintentional hand movements of the target object. Since the gesture recognition result is the number of fingers extended by the user, the processing complexity of gesture recognition is reduced, control efficiency is improved, and the operation of the projection display device becomes more intuitive. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0013] Figure 1 This is a schematic diagram of the structure of a projection display device disclosed in an embodiment of this application;

[0014] Figure 2 This is a schematic diagram of the imaging principle of a TOF sensor disclosed in an embodiment of this application;

[0015] Figure 3 This is a schematic diagram of the system architecture of a projection display device disclosed in an embodiment of this application;

[0016] Figure 4 This is a flowchart illustrating a gesture control method for a projection display device disclosed in an embodiment of this application;

[0017] Figure 5 This is a schematic diagram of a target depth image in the gesture control method disclosed in the embodiments of this application;

[0018] Figure 6 This is a schematic diagram illustrating the recognition of the number of fingers in a hand region in the gesture control method disclosed in this application embodiment;

[0019] Figure 7 This is a flowchart illustrating a gesture control method disclosed in this application for detecting whether the hand and head regions meet the gesture recognition conditions.

[0020] Figure 8 This is a flowchart illustrating a gesture control method disclosed in this application for detecting whether the head region meets the gesture recognition conditions;

[0021] Figure 9 This is a flowchart illustrating a gesture control method disclosed in this application for detecting whether the hand area and torso area meet the gesture recognition conditions.

[0022] Figure 10 This is another schematic diagram of the target depth image in the gesture control method disclosed in the embodiments of this application;

[0023] Figure 11 This is a flowchart illustrating the determination of a target operation mapping relationship in the gesture control method disclosed in this application embodiment;

[0024] Figure 12 This is a schematic diagram of a main control circuit disclosed in an embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0027] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0028] It should be noted that the terms "first, second, third" used in the embodiments of this application are used to distinguish similar or different objects and do not represent a specific order of objects. It can be understood that "first, second, third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0029] As the concept of human-computer interaction continues to deepen and develop, market demands are also changing. People are increasingly eager to break free from the constraints of hardware devices and achieve more intelligent and natural control methods. Among these, gesture recognition technology has attracted much attention due to its intuitiveness and convenience. When enjoying movies or other entertainment activities on projection display devices, users no longer need to be distracted by searching for remote controls or issuing voice commands, thus avoiding the inconvenience caused by these actions.

[0030] However, gesture control still faces certain challenges in application. Unlike users intentionally controlling devices like phones or tablets with air gestures, projection display devices are more likely to capture images of the user's hand area in usage scenarios. If the user's intention cannot be accurately determined, unintentional actions such as waving, raising the hand to pick up an item, or yawning may be recognized as control commands by the projection display device, causing unnecessary reactions, affecting the user's normal experience, and causing inconvenience.

[0031] Therefore, after acquiring an image containing the user's hand area, the projection display device needs to correctly determine which gestures are the user's controlled gestures and which are unintentional actions to reduce the risk of accidental device manipulation. Some projection display recognition methods reduce the possibility of accidental manipulation by setting more complex gestures. However, since projection display devices are usually used in low-light environments, they require Time-of-Flight (TOF) sensors for image acquisition. But the image accuracy of this method is often lower than that of traditional color camera modules, which may affect the accuracy of gesture recognition. In addition, complex gesture recognition algorithms also place higher demands on the device's processing power.

[0032] In view of this, embodiments of this application provide a projection display device that analyzes and identifies target depth images acquired by a TOF sensor. This analysis goes beyond the hand area; it incorporates multi-dimensional information such as the head and torso areas as target areas for detecting and preventing accidental manipulation. Compared to detecting and identifying only the hand area, this introduces more reference information, allowing for a more accurate determination of the user's intentions. After the main control circuit determines that the target area meets the gesture recognition criteria (i.e., the user's manipulation is not accidental), gesture recognition is then performed on the hand area. The result of gesture recognition is the number of extended fingers of the target object, reducing the complexity of the recognition algorithm and thus improving the response speed and efficiency of gesture control. Even when the clarity of the target depth image is relatively low (TOF sensor accuracy is low), it can stably and accurately recognize user gestures, providing users with a stable and reliable interactive experience.

[0033] To make the purpose and technical solution of this application clearer and more intuitive, the projection display device disclosed in this application will be described in detail below with reference to the accompanying drawings.

[0034] It should be noted that the projection display device provided in the application embodiments can have various implementation forms. For example, according to the light source, it can be a laser projection device, an LED projection display device, etc. According to the installation form, it can be a ceiling-mounted projection display device (usually installed on the ceiling), a desktop projection display device (which can also be installed by a bracket, etc.). This application does not limit it in this regard.

[0035] Please see Figure 1 , Figure 1 This is a schematic diagram of the structure of a projection display device disclosed in an embodiment of this application. Figure 1 The projection display device 10 shown includes a projection module 11, a TOF sensor 12, and a main control circuit 13.

[0036] The projection module 11 may include a light source, a light modulation component, and a lens, etc. Figure 1 (Not shown in the image), the projection display device can project the image onto a screen or wall through the projection module 11, which is not limited here. The TOF sensor 12 can be used to measure the distance between the device and the projection surface to realize the automatic focusing function of the projection module 11, and the ambient light sensor can be used to monitor the lighting conditions of the surrounding environment in real time to ensure that the projection display device 10 can automatically adjust parameters such as brightness and contrast according to the light intensity to maintain the best visual effect.

[0037] It should be noted that the TOF sensor 12 is equipped with an active light source, making it less affected by ambient light. This overcomes the limitations of relying solely on ambient light to capture color images, allowing it to acquire depth images under any ambient light conditions. Please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram illustrating the imaging principle of a TOF sensor disclosed in an embodiment of this application. The TOF sensor 12 can transmit modulated light signals of continuous waves forward through its transmitter (active light source). These signals propagate in space and are reflected back after encountering an object, where they are captured by a receiver. By comparing the phase difference or time-of-flight difference between the transmitted and received signals, the round-trip time of the light signal between the sensor and the object can be accurately calculated, and the actual distance (depth) from the sensor to the object can be calculated based on the speed of light. By transmitting and receiving light signals at multiple points in space, a complete depth image can be obtained.

[0038] In some possible embodiments, the projection display device may employ a TOF sensor 12 with a resolution of 320*240, 640*480, or other resolutions to recognize changes in the user's gestures, which is not limited here.

[0039] In some possible embodiments, the TOF sensor 12 is positioned on the side of the projection module 11 in the projection display device 10 where the light signal is projected, and the acquisition area of ​​the TOF sensor 12 overlaps with the projection area of ​​the projection module 11. In this case, when the ranging function of the projection display device 10 is enabled, the TOF sensor 12 can be used to assist in obtaining the distance and angle between the projection display device 10 and projection surfaces such as walls and screens, so as to adjust the light signal emitted by the projection module 11 and achieve a better display effect. When the gesture recognition function of the projection display device is enabled, the TOF sensor 12 can acquire a depth image, and the main control circuit 13 can perform gesture recognition based on the depth image to control the projection display device to perform operations corresponding to the user's gestures.

[0040] In addition, some projection display devices 10 also have eye protection functions. When the eye protection function of the projection display device 10 is enabled, the main control circuit 13 determines the target screen area in the projection screen of the projection display device 10 according to the pixel coordinates of the head area or user area included in the target depth image collected by the TOF sensor 12, and reduces the pixel brightness of the target screen area to reduce the potential risk to the user's eyes when the projection module 11 sends light signals.

[0041] It should be noted that, given the diverse installation locations of projection display devices, users can flexibly choose to install the device to the side, in front of, or above their seat, depending on their actual needs. To ensure that the TOF sensor on the projection display device can effectively acquire depth images including the user's hand, head, or torso areas under these different installation conditions, and to determine whether to perform gesture recognition based on these areas, a wide-angle TOF sensor can be used. Its wide field of view can cover a wider area, thereby increasing the possibility of capturing the user's hands and head. Alternatively, additional TOF sensors can be added at appropriate locations on the device. By using a multi-sensor layout, the limitations of a single sensor in terms of field of view can be compensated for, ensuring that the user can be effectively monitored by at least one TOF sensor regardless of their position. Furthermore, the method of prompting the user to install the projection display device in a designated location during installation is not limited here.

[0042] In order to project images or videos and enable gesture control based on depth images captured by a Time-of-Flight (TOF) sensor, projection display devices require their various components to operate according to pre-defined control logic. Therefore, to further understand the relationships between the components of a projection display device, please refer to [link / reference needed]. Figure 3 , Figure 3 This is a schematic diagram of the system architecture of a projection display device disclosed in an embodiment of this application. For example... Figure 3 The projection display device shown is a laser projection display device, which may include, but is not limited to, a main control circuit 301, a TOF sensor 302, a display driver circuit 303, an HDMI (interface) 304, a USB (interface) 305, a DMD (Digital Micrometer Device) 306, a galvanometer 307, a lens 308, a laser driver circuit 309, a laser 310, and a power supply circuit 311. The system architecture of the projection display device may include more or fewer components than shown, or combine certain components, or arrange different components.

[0043] In some possible embodiments, the projection display module provided in this application includes, but is not limited to, a display driving circuit 303, a DMD (Digital Micrometer Device) 306, a galvanometer 307, a lens 308, a laser driving circuit 309, and a laser 310.

[0044] The main control circuit 301, TOF sensor 302, and display driver circuit 303 are powered by the power supply circuit 311. The TOF sensor is indirectly powered through the main control circuit 301. The main control circuit 301 can transmit VBO (V-By-One) signals (a signal used in video signal transmission) to the display driver circuit. After receiving the VBO signal, the display driver circuit processes the signal and then transmits red, green, and blue PWM (Pulse Width Modulation) signals (a pulse width modulation signal that simulates signal changes by changing the duty cycle) and enable information (a control signal used to enable or disable specific functions or operations) to the laser driver circuit 309 to control the laser 310 to generate laser light. At the same time, the display driver circuit also controls the DMD 306 (a spatial light modulator that uses a micromirror array to control the reflection of light) and the galvanometer 307 (an optical instrument that deflects the light beam) to adjust the laser light generated by the laser 310 so as to project a clear video or image.

[0045] In some possible embodiments, the main control circuit 301 can receive video or image signals from the corresponding HDMI peripheral or USB peripheral via HDMI (interface) 304 or USB (interface) 305, convert them into VBO signals and transmit them to the display driver circuit 303, thereby realizing the projection of the image of the projection display device.

[0046] In some possible embodiments, when the TOF sensor 302 acquires a target depth image containing the hand and head of the target object (user), the depth image is transmitted to the main control circuit 301, and the main control circuit 301 performs a series of gesture recognitions on the depth image. Based on the recognition results, the VBO signal is adjusted to control the projection screen of the projection display device, thereby achieving effects such as controlling video fast forward and pause.

[0047] The above introduction explains the working principles of each component of a projection display device and the control logic of the projection display. To ensure that the projection display device can determine the user's operating intentions and control the device based on gestures, the main control circuit 301 needs to detect whether the target area contained in the depth image acquired by the TOF sensor 302 meets the gesture recognition conditions, and perform gesture recognition processing on the hand area if the conditions are met.

[0048] This application provides a gesture control method for a projection display device. Please refer to... Figure 4 , Figure 4 This is a flowchart illustrating a gesture control method for a projection display device disclosed in an embodiment of this application. The gesture control method may include the following steps:

[0049] Step 401: When the main control circuit determines that the gesture recognition function of the projection display device is enabled and the target area included in the target depth image acquired by the TOF sensor meets the gesture recognition conditions, it performs gesture recognition on the hand area included in the target depth image, obtains the number of fingers extended by the target object, and controls the projection display device to perform the operation corresponding to the number of fingers according to the preset target operation mapping relationship.

[0050] It should be noted that the target area includes the head area, or the target area includes the head area and the hand area, or the target area includes the torso area and the hand area; the target operation mapping relationship includes the mapping relationship between the preset number of fingers and the preset operation of the projection display device, and the gesture recognition result includes the number of fingers extended by the object.

[0051] In the embodiments of this application, the gesture recognition function of the projection display device can be set to be enabled by default, or the user can turn this function on or off through a remote control, mobile phone or other control device, which is not limited here.

[0052] In this embodiment, the TOF sensor can acquire depth images in real time according to its sampling frequency, or acquire depth images according to a preset time interval, and then transmit these depth image data to the main control circuit for detection to determine whether the depth image includes the head and hands of the target object. The determination can be made according to the data processing capability of the main control circuit and the user's requirements for gesture sensitivity, and is not limited here.

[0053] In some possible embodiments, the main control circuit can use a preset detection algorithm to detect hands and heads in the images acquired by the TOF sensor. If it is determined that both the hand and head of the target object are present in the target depth image, the main control circuit will then proceed to the next step of detecting whether the target area meets the hand gesture recognition criteria. The preset detection algorithm can be a network model algorithm such as YOLOv5 or VGG16, or it can be a traditional computer vision algorithm such as shape analysis or template matching; no limitation is made here.

[0054] It should be noted that the order of head detection and hand detection can be sequential or simultaneous, depending on the data processing capabilities of the main control circuit, and is not limited here. By detecting whether the target depth image contains a hand region, it can be determined whether the target depth image can be used for gesture recognition. Detecting whether it contains a head region determines whether the target depth image contains sufficient information to decide whether to recognize the hand region. This judgment mechanism can prevent the projection display device from incorrectly recognizing unintentional hand movements of the user, avoiding unnecessary responses and thus improving the user experience. If it is determined that the target depth image acquired by the TOF sensor includes the target object's hand and head, the main control circuit can segment the target region in the target depth image to determine whether it meets the conditions for gesture recognition.

[0055] It is understandable that users may unintentionally make gestures when using projection display devices, which are captured by TOF sensors. If only the hand area is used for anti-accidental manipulation detection, the information from the hand area alone may not be sufficient to accurately determine the user's intention, as this ignores other important information related to the user's gesture. In this application, anti-accidental manipulation detection is performed based on the target area, provided that the target depth image contains both a hand (information required for gesture recognition) and a head (information helpful for accidental manipulation detection).

[0056] In some possible embodiments, the target region may include the hand region corresponding to the hand of the target object and the head region corresponding to the head in the target depth image. When setting gesture recognition conditions, the pixel distance between the head region and the hand region, or their relative positional relationship in depth, can be considered as the basis for judgment.

[0057] For example, the extension of a user's hand and arm can be determined based on the pixel distance between the hand and head regions, such as the pixel distance between the center points of the regions, the minimum pixel distance between the two regions, or the maximum pixel distance between the two regions. It is understood that, with constant depth information, a larger pixel distance between the hand and head regions indicates a more extended hand (or arm) of the target object. Therefore, in some embodiments, a preset distance reference value can be set to determine whether the user's hand (or arm) is in a specified extension state and whether the user's gesture is used to control the projection display device.

[0058] Optionally, gesture recognition conditions can be set based on the relative depth positions of the hand and head regions. For example, if the difference between the depth information of the hand and head (e.g., the depth value at the center of the region or the average depth of the region) is large and exceeds a set threshold, the target object may be picking up an item forward or backward. In this case, it can be determined that the target object was accidentally manipulating the image when the depth image of the target was captured.

[0059] In some possible embodiments, the target area may also include the head region. Since the accuracy of TOF sensors is typically lower than that of color sensors, and there is a certain distance (usually greater than 1 meter) between the target object (user) and the projection display device, it is not suitable to use a method based on human eye gaze to determine whether the user has unintentionally performed an operation. However, the pixel area error of the head region is relatively smaller than that of the eye. Therefore, in some embodiments, the size of the head region can be used to determine whether the target object is facing the TOF sensor when making a gesture, and thus whether gesture recognition should be performed on the hand region in the target depth image.

[0060] In some possible embodiments, the target area may also be the torso area and the hand area. It is understood that when the target depth image is determined to include the head and hands, in addition to characteristic forms such as the user entering the TOF sensor's acquisition area by looking up, the target depth image will typically also include the torso area of ​​the target object. Therefore, in some embodiments, the relative positional relationship between the torso area and the hand area can be used to determine whether to recognize the user's gesture.

[0061] For example, gesture recognition conditions can be set based on the relative depth relationship between the torso region and the hand region in the target depth image to determine whether to recognize the user's gesture. It is understandable that when a user performs actions such as patting their chest, in related technologies, if the target region (torso region and hand region) is not determined to meet the gesture recognition conditions, the main control circuit may perform gesture recognition on the hand region, resulting in a gesture recognition result of the target object extending five fingers, and control the projection display device to perform operations corresponding to the number of fingers. This may lead to responses from the projection display device such as adjusting playback speed or pausing, affecting the user experience. Therefore, in some embodiments, when the depth difference between the hand region and the head region is small (less than a set depth value), it can be determined that the target region contained in the target depth image does not meet the gesture recognition conditions. In this case, gesture recognition is not performed on the hand region in the target depth image, improving the user experience.

[0062] In some possible embodiments, the target area includes a head area, a hand area, and a torso area. Referring to the foregoing embodiments, at least two of the following can be selected: the head area, or the head area and the hand area, or the torso area and the hand area, to set the gesture recognition conditions. This enhances the accuracy and strictness of detecting whether a user intends to manipulate the projection display device, making it suitable for application scenarios with high precision requirements.

[0063] It should be noted that in this application, when segmenting the target region in the target depth image, the same algorithm used by the main control circuit for detecting hands or heads can be used. These algorithms may include traditional image processing methods or deep learning-based methods, and are not limited thereto.

[0064] In some possible embodiments, when the target region includes at least one of a hand region or a head region, the main control circuit can segment the hand region and / or head region when detecting whether the target depth image contains the head and hand of the target object. This segmentation method helps improve the efficiency of image segmentation processing, avoids repeatedly detecting and segmenting the same image region multiple times, reduces the consumption of computing resources, and improves the response speed of gesture recognition.

[0065] In some possible embodiments, the target region obtained by segmenting the target depth image is a rectangular region. See also... Figure 5 , Figure 5 This is a schematic diagram of a target depth image in the gesture control method disclosed in an embodiment of this application. For example... Figure 5 As shown, in the target depth image, the three selected areas correspond to the hand area, head area, and torso area, respectively. Figure 5 This is just an example and does not limit the selected target area.

[0066] In some possible embodiments, the target depth image acquired by the TOF sensor includes two hand regions corresponding to the two hands of the target object. To avoid the inability to control the projection display device according to the user's gestures when the gestures of the left and right hand regions are different, the hand regions located on the left or right side of the target depth image can be selected for anti-mistouch detection and gesture recognition to improve the accuracy of gesture recognition. (Note that the left and right in the image may be mirror images of the left and right of the target object in reality, and can be selected according to actual needs.)

[0067] In this embodiment of the application, after the main control circuit determines that the target area included in the target depth image acquired by the TOF sensor meets the gesture recognition conditions, the main control circuit performs gesture recognition on the hand area.

[0068] In this application, since projection display devices are often used in low-light environments to obtain better display effects, TOF sensors with active light sources are usually used to acquire target depth images. Compared with traditional color camera modules, such TOF sensors often have certain limitations in image accuracy (resolution). Therefore, the number of fingers extended by the target object, rather than complex hand gestures (the action of combining fingers into a specific shape), can be used as the gesture recognition result, which helps to improve the accuracy and stability of gesture recognition.

[0069] In some possible embodiments, the main control circuit performs gesture recognition on the hand area to obtain the number of outstretched fingers of the target object, including:

[0070] The main control circuit performs gesture recognition on the hand area according to a preset curvature detection algorithm to obtain the number of fingers extended by the target object.

[0071] It should be noted that the preset curvature detection algorithm can be the K-Curvature algorithm.

[0072] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating the recognition of the number of fingers in a hand region within the gesture control method disclosed in this application. For example... Figure 6 As shown, with the preset curvature detection algorithm being the K-Curvature algorithm, the fingertip point A is first found, and then the previous K points (point B) and the next K points (point C) are found based on point A. At this point, points A, B, and C can generate vectors. Recorded as and Given an angle α, the cosine of the angle can be obtained using the following cosine values:

[0073]

[0074] If α is less than 90°, then point A can be identified as the fingertip; conversely, if α is greater than 90°, then point A is not identified as the fingertip. Therefore, the number of outstretched fingers of the target object in the hand region can be obtained through this preset curvature detection algorithm.

[0075] In this embodiment of the application, after the main control circuit obtains the number of fingers extended by the target object, it can control the projection display device to perform the operation corresponding to the number of fingers according to the number of fingers and the preset target operation mapping relationship.

[0076] It should be noted that a user typically has five fingers on one hand. Taking the extension of 0 to 5 fingers as an example, this corresponds to six different operations on the projection display device. However, when the user extends 0 fingers, i.e., makes a fist, the hand may appear as a large and relatively uniform brightness area in the depth image, making it difficult to recognize the hand area. Therefore, in some embodiments, the target operation mapping relationship may include the mapping relationship between extending 1 finger (number of fingers is 1) to extending 5 fingers (number of fingers is 5) and the five operations that the projection pixel device can perform.

[0077] In some possible embodiments, the projection display device includes multiple preset operation mapping relationships, different preset operation mapping relationships correspond to different preset projection content of the projection display device, the projection content includes video content or non-video content, and the projection display device further includes:

[0078] The main control circuit acquires the current projected content of the projection display device and determines the target operation mapping relationship as the preset operation mapping relationship that corresponds to the current projected content among multiple preset operation mapping relationships.

[0079] It should be noted that since the target operation mapping relationship is determined based on the current projected content of the projection display device, when the projection display device projects different content, the target object (i.e., the user) can trigger different operation effects using the same gesture.

[0080] For example, when the projected content includes video content or non-video content, if the main control circuit obtains that the current projected content of the projection display device is video content, the target operation mapping relationship determined according to the projected content can be as shown in Table 1. Different numbers of fingers correspond to different operations in the video playback scenario.

[0081] Table 1

[0082] Number of fingers The corresponding operations in the target operation mapping relationship 1 1x playback speed 2 2x playback speed 3 3x playback speed 4 Playback paused 5 Start playing

[0083] When the main control circuit obtains that the current projection content of the projection display device is non-video content (for example, the projection display device displays the main interface or other interfaces unrelated to video playback), the target operation mapping relationship determined according to the projection content can be shown in Table 2. Different numbers of fingers correspond to different operations in general scenarios.

[0084] Table 2

[0085] Number of fingers The corresponding operations in the target operation mapping relationship 1 Return to main interface 2 Return to the previous screen 3 Increase display brightness 4 Reduce display brightness 5 Turn sound off / on

[0086] Understandably, with technological advancements, the functions of projection display devices are no longer limited to simply receiving external signals. Instead, they are integrating various applications, particularly video applications from different platforms, aiming to provide users with more diverse choices. The gesture control method disclosed in this application establishes a relationship between operation mapping and projection content. Thus, when the main control circuit detects that a target object is projecting video through any video application (i.e., the projected content is video content), it determines the target operation mapping to be one of several preset operation mappings corresponding to the video content (as shown in Table 1). In this way, projection display device manufacturers do not need to separately adapt gesture control functions for different video applications; users can achieve gesture control of different video applications through the device's own control functions.

[0087] In some possible embodiments, before the main control circuit performs gesture recognition on the hand area, the projection display device further includes:

[0088] The main control circuit acquires the depth information corresponding to the hand area and determines the recognition plane based on the depth information. The depth information includes the maximum or minimum depth value corresponding to the hand contour of the target object in the hand area.

[0089] If the difference between the depth value of each pixel in the determined hand contour and the depth of the recognition plane is greater than a preset error threshold, the main control circuit controls the projection display device to output prompt information, which is used to prompt the target object to adjust the hand posture.

[0090] It should be noted that, as Figure 5 As shown, the hand region can be a rectangular area, containing a hand outline (represented by a light color) and a background (represented by a dark color). The main control circuit obtains the hand outline through edge detection and other methods, and obtains the maximum or minimum depth value corresponding to multiple pixels contained in the hand outline. The recognition plane is determined by either the maximum or minimum depth value.

[0091] After determining the recognition plane, the main control circuit evaluates the depth difference between each pixel in the hand contour and the recognition plane. If it finds that the depth difference between a pixel and the recognition plane exceeds a preset error threshold, it means that when the TOF sensor acquires the target depth image, the target object's hand posture may be significantly offset relative to the vertical recognition plane, potentially leading to errors in gesture recognition results—that is, the main control circuit may identify finger values ​​that do not match the actual values. Therefore, it is necessary to control the projection display device to output prompts to prompt the target object to adjust its hand posture, ensuring that the projection display device can correctly respond to the target object's gestures.

[0092] In some possible embodiments, the preset error threshold is determined based on the thickness of a person's hand (palm). For example, the thickness of a person's palm is usually around 3cm. In order to retain a certain offset margin, the error threshold can be set to 5cm, or other values ​​can be set according to sensitivity requirements. No limitation is made here.

[0093] For example, if the minimum depth value corresponding to the hand contour from the TOF sensor is 2m, and the preset error threshold is 5cm, then the recognition plane can be determined to be the plane 2m away from the TOF sensor. If all pixels in the hand contour of the target image are between 200cm and 205cm (i.e., there are no pixels in the hand contour whose depth value differs from the depth of the recognition plane by more than the preset error threshold), then gesture recognition can be performed on the hand area, and the projection display device can be controlled based on the gesture recognition result. Otherwise, a prompt message needs to be output until the target object adjusts its hand posture, and then gesture recognition is performed based on the newly acquired depth image.

[0094] In some possible embodiments, the projection display device further includes:

[0095] The main control circuit determines the target image area in the projection screen of the projection display device based on the pixel coordinates of each pixel in the head area, and reduces the pixel brightness of the target image area.

[0096] It should be noted that when using a projection display device, factors such as improper installation or the user walking in front of the projection light source may cause the user's eyes to be illuminated by the light, resulting in eye discomfort or even damage to the user's vision. To avoid this, the main controller can determine the target area in the projected image based on the pixel coordinates of the head region, including the eyes, and reduce the pixel brightness of the target area. This reduces the intensity of light shining into the user's eyes, effectively protecting the user's eyes during gesture recognition.

[0097] In some possible embodiments, the correspondence between the head region and the target image region in the projected image of the projection display device is determined based on the placement of the TOF sensor and the projection module (including but not limited to the projection light source, lens, etc.) on the projection display device. After determining the positional relationship between the TOF sensor and the projection module, an image mapping relationship can be constructed between the image region in the depth image and the image region in the projected image of the projection display device. The main control circuit can then obtain the target image region in the projected image of the display device corresponding to the head region based on the determined image mapping relationship, ensuring that the projection display device does not adversely affect the user's eyes during gesture recognition and daily use.

[0098] The above embodiments describe a gesture control method applied to projection display devices. When the gesture recognition function of the projection display device is enabled, a TOF sensor is used to acquire a target depth image. The main control circuit acquires the target area in the target depth image and detects whether the target area meets the gesture recognition conditions, reducing the risk of accidental operation of the projection display device by the user. After the main control circuit confirms that the target area meets the gesture recognition conditions, the main control circuit performs gesture recognition on the hand area, and the recognition result is the number of the user's fingers, thereby improving recognition efficiency and making the operation of the projection display device more intuitive.

[0099] The following embodiments further illustrate how the main control circuit detects whether the target area meets the gesture recognition conditions when the target area includes both the head area and the hand area.

[0100] Please see Figure 7 , Figure 7 This is a flowchart illustrating a gesture control method disclosed in this application for detecting whether the hand and head regions meet the gesture recognition conditions, which may include the following steps:

[0101] In some possible embodiments, the target region includes a head region and a hand region. The gesture recognition condition includes that the pixel distance between the pixel coordinates of the center pixel of the head region and the pixel coordinates of the center pixel of the hand region is less than a first distance threshold, which is determined based on the depth value of the center pixel of the head region and / or the depth value of the center pixel of the hand region.

[0102] Step 701: When the main control circuit determines that the gesture recognition function of the projection display device is enabled, it performs segmentation processing on the target depth image to obtain the hand region and head region in the target depth image.

[0103] In some possible embodiments, the main control circuit performs segmentation processing on the target depth image to obtain the target region in the target depth image, including:

[0104] The main control circuit performs segmentation processing on the target depth image to obtain the hand and head regions in the target depth image.

[0105] Step 702: The main control circuit obtains the pixel coordinates of the center pixel of the head region and the pixel coordinates of the center pixel of the hand region, calculates the pixel distance between the pixel coordinates of the center pixel of the head region and the pixel coordinates of the center pixel of the hand region, and determines a first distance threshold based on the depth value of the center pixel of the head region and / or the depth value of the center pixel of the hand region.

[0106] In some possible embodiments, the main control circuit obtains the pixel distance between the pixel coordinates of the center pixel of the head region and the pixel coordinates of the center pixel of the hand region, and determines a first distance threshold based on the depth value of the center pixel of the head region and / or the depth value of the center pixel of the hand region, so as to determine whether the target region, i.e. the head region and the hand region, meets the gesture recognition conditions.

[0107] It is understandable that, when the actual distance between the target object's hand and head remains constant, the pixel distance between the center pixels of the hand region and the head region is closely related to the actual distance between the TOF sensor and the target object. When the depth values ​​of the center pixels of the head region and the hand region remain constant, a larger pixel distance indicates a greater actual distance between the target object's hand and head (e.g., when a user reaches out to pick up or pass an item). Based on the pixel distance between the center pixel coordinates of the head region and the center pixel coordinates of the hand region, it is possible to infer whether the target object intends to manipulate the projection display device, thus preventing the device from misidentifying unintentional hand movements and affecting the user experience.

[0108] In some possible embodiments, the distance difference between the target object's hand and head caused by the hand extending forward or swinging backward is relatively small compared to the distance between the target object and the projection display device. The first distance threshold can be determined by either the depth value of the center pixel of the head region or the depth value of the center pixel of the hand region in the target depth image.

[0109] For example, the depth values ​​of the center pixels of the head region and the center pixels of the hand region can be multiplied by a preset coefficient to determine the product as the first distance threshold. The preset coefficient can be determined through multiple experiments or expert evaluation, and is not limited here.

[0110] In some possible embodiments, a baseline depth value can be calculated by averaging the depth values ​​of the center pixels of the head region and the center pixels of the hand region in the target depth image. This baseline depth value is then multiplied by a preset coefficient, and the resulting product can be used as the first distance threshold.

[0111] In some possible embodiments, the sensitivity of determining whether a target region meets the gesture recognition criteria can be controlled by adjusting the value of a preset coefficient to prevent false recognition. When the depth of the center pixels in the hand and head regions remains constant, a larger preset coefficient means a larger first distance threshold is needed to determine whether the target region meets the gesture recognition criteria.

[0112] After determining the first distance threshold, the pixel distance between the center pixel coordinates of the head region and the center pixel coordinates of the hand region can be compared with the first distance threshold. If the pixel distance is less than the first distance threshold, the target region is judged to meet the gesture recognition conditions, and gesture recognition is performed on the hand region.

[0113] In some possible embodiments, the pixel distance between the pixel coordinates of the center pixel of the head region and the pixel coordinates of the center pixel of the hand region can be set to be greater than or equal to a first distance threshold to determine that the target region meets the gesture recognition conditions. This can be flexibly adjusted according to actual needs and the selection of preset coefficients, and is not limited here.

[0114] Step 703: If the hand area and head area meet the gesture recognition conditions, the main control circuit performs gesture recognition on the hand area, obtains the number of fingers extended by the target object, and controls the projection display device to perform the operation corresponding to the number of fingers according to the preset target operation mapping relationship.

[0115] The above embodiments describe a method for determining whether a target region meets the conditions for gesture recognition based on the head region and the hand region, and then determining whether to perform gesture recognition on the hand region. By detecting the pixel distance between the center pixel coordinates of the hand region and the center pixel coordinates of the head region in the target depth image, the extension and contraction state of the target object's hand can be inferred, so as to avoid the projection display device responding to the user's unintentional hand movements.

[0116] The following embodiments further illustrate how the main control circuit detects whether the target area meets the conditions for gesture recognition when the target area includes the head area.

[0117] Please see Figure 8 , Figure 8 This is a flowchart illustrating a gesture control method disclosed in this application for detecting whether the head region meets the gesture recognition conditions, which may include the following steps:

[0118] In some possible embodiments, the target region includes a head region, and the gesture recognition condition includes that the pixel area of ​​the head contour is greater than or equal to an area threshold. The head contour is obtained by the main control circuit based on the head region, and the area threshold is determined based on the depth value of the center pixel of the head region and a preset area mapping relationship. The area mapping relationship includes a preset depth value and a preset area mapping relationship.

[0119] Step 801: When the main control circuit determines that the gesture recognition function of the projection display device is enabled, it performs segmentation processing on the target depth image to obtain the head region in the target depth image.

[0120] In some possible embodiments, the main control circuit performs segmentation processing on the target depth image to obtain the target region in the target depth image, including:

[0121] The main control circuit performs segmentation processing on the target depth image to obtain the head region in the target depth image.

[0122] Step 802: The main control circuit obtains the pixel area of ​​the head contour in the head region and the depth value of the center pixel of the head region.

[0123] In some possible embodiments, when the target region is the head region in the target depth image, the depth value of the center pixel of the head region and the pixel area of ​​the head contour are obtained.

[0124] It should be noted that in this application, the head region can be a rectangular region or other suitable shape. In addition to the image content of the target object's head, the head region may also contain background content, such as sofa backrest, ceiling, etc. The head contour in the head region can be identified by edge detection algorithms and other methods, and the pixel area of ​​the head contour can be obtained.

[0125] In some possible embodiments, in addition to edge detection algorithms, the depth value of the center pixel of the head region can be obtained and combined with a preset depth threshold to determine the pixels in the head region whose depth difference with the center pixel is less than the preset depth threshold as pixels of the head contour, and the pixel area of ​​the head contour can be obtained.

[0126] It should be noted that the center pixel in the head region is a point in the head contour, and the depth difference between pixels in the head contour is usually smaller than the difference between pixels in the head contour and the background in the target region. Therefore, the head contour can be quickly identified by setting a depth threshold. The preset depth threshold can be determined by calculating the depth distribution between pixels in the sample head contour in the sample image depth image.

[0127] Step 803: The main control circuit determines the area threshold based on the depth value of the center pixel of the head region and the preset area mapping relationship. The preset area mapping relationship includes the mapping relationship between the preset depth value and the preset area.

[0128] In some possible embodiments, after the main control circuit calculates the depth value of the center pixel of the head region corresponding to the head region and the pixel area of ​​the head contour, it can compare the pixel area of ​​the head contour with the size of an area threshold. If the pixel area of ​​the head contour is greater than or equal to the area threshold, it determines that the target object's head is facing the TOF sensor (projection display device) and that the target object intends to control the projection display device. If the pixel area of ​​the head contour is less than the area threshold, it determines that the target object's head is not facing the TOF sensor (projection display device). Since the head is deflected, the pixel area is less than the area threshold, and therefore it is determined that the target object does not intend to control the projection display device, and there is no need to perform gesture recognition on the hand area.

[0129] It should be noted that the area threshold is determined based on the depth value of the center pixel in the head region and a preset area mapping relationship, which includes a preset depth value and a preset area mapping relationship. This ensures that when the target object is at different distances from the TOF sensor (projection display device), an area threshold adapted to the current distance is obtained, thereby reducing the risk of the projection display misrecognizing user gestures.

[0130] Step 804: If the head region meets the gesture recognition conditions, the main control circuit acquires the hand region in the target depth image, performs gesture recognition on the hand region, acquires the number of fingers extended by the target object, and controls the projection display device to perform the operation corresponding to the number of fingers according to the preset target operation mapping relationship.

[0131] In some possible embodiments, after the main control circuit determines that the head region meets the gesture recognition conditions, the projection display device further includes: the main control circuit acquiring the hand region in the target depth image.

[0132] The above embodiments describe a method for determining whether a target region meets the conditions for gesture recognition based on the head region, and then determining whether to perform gesture recognition on the hand region. By detecting the pixel area of ​​the head contour in the head region of the target depth image, it is inferred whether the head of the target object is facing the TOF sensor (projection display device), so as to avoid the projection display device responding to the user's unintentional hand movements.

[0133] The following embodiments further illustrate how the main control circuit detects whether the target area meets the gesture recognition conditions when the target area includes both the hand area and the torso area.

[0134] Please see Figure 9 , Figure 9 This is a flowchart illustrating a gesture control method disclosed in this application for detecting whether the hand area and torso area meet the gesture recognition conditions, which may include the following steps:

[0135] In some possible embodiments, the target area includes a torso area and a hand area. The gesture recognition conditions include that the hand area is located within the torso area, and the difference between the average depth of each pixel in the hand contour and the average depth of each pixel outside the hand contour is greater than or equal to a preset depth threshold. The hand contour is obtained by the main control circuit based on the hand area.

[0136] Step 901: When the main control circuit determines that the gesture recognition function of the projection display device is enabled, it performs segmentation processing on the target depth image to obtain the hand region and torso region in the target depth image.

[0137] In some possible embodiments, the main control circuit performs segmentation processing on the target depth image to obtain the target region in the target depth image, including:

[0138] The main control circuit performs segmentation processing on the target depth image to obtain the hand region and torso region in the target depth image.

[0139] Step 902: The main control circuit determines whether the hand area is within the torso area.

[0140] In some possible embodiments, it is determined whether the hand region is within the torso region if all pixels in the hand region are within the torso region.

[0141] Please see Figure 10 , Figure 10 This is another schematic diagram of the target depth image in the gesture control method disclosed in the embodiments of this application. For example... Figure 10 As shown, all pixels in the hand region are within the torso region, which can determine whether the hand region is within the torso region and proceed to the next step 903.

[0142] In some possible embodiments, if some pixels in the hand region are located within the torso region, and the proportion of pixels located within the torso region to the total number of pixels in the hand region is greater than a preset proportion threshold, it is determined whether the hand region is located within the torso region.

[0143] It should be noted that when the hand region partially overlaps with the torso region, the presence of the hand region within the torso region can be determined by the proportion of the hand region's pixels to the total pixels of the hand region. For example, a preset proportion threshold of 90% can be used.

[0144] Step 903: When the main control circuit determines that the hand region is within the torso region, it obtains the average depth of each pixel in the hand contour of the target object in the hand region and the average depth of each pixel in the hand region excluding the hand contour.

[0145] It should be noted that, assuming the hand region is within the torso region, the average depth of each pixel in the hand contour of the target object within the hand region can represent the distance between the target object's hand and the TOF sensor. The pixels in the hand region excluding the hand contour represent the pixels in the torso region surrounding the hand contour, and the average depth of these pixels can represent the distance between the torso surrounding the target object's hand and the TOF sensor.

[0146] The difference between the average depth of each pixel within the hand contour and the average depth of each pixel outside the hand contour can be used to characterize the actual distance between the target object's hand and torso. When a user performs a gesture, they typically extend their hand a certain distance. Therefore, if the main control circuit determines that the difference between the average depth of each pixel within the hand contour and the average depth of each pixel outside the hand contour is less than a preset depth threshold, the target object's hand and torso are nearly touching, possibly indicating a resting action such as patting their chest or placing their hand on their torso. In this case, it can be determined that the target object's hand movement is not an attempt to control the projection display device, and the main control circuit does not perform gesture recognition on the hand area. Conversely, if the difference between the average depth of each pixel within the hand contour and the average depth of each pixel outside the hand contour is greater than or equal to the preset depth threshold, it is determined that the target object intends to control the projection display device, and gesture recognition is performed on the hand area.

[0147] For example, since the thickness of a user's palm is usually around 3cm, the preset depth threshold can be set to 5cm or other values, which are not limited here.

[0148] Step 904: If the hand area and torso area meet the gesture recognition conditions, the main control circuit performs gesture recognition on the hand area, obtains the number of fingers extended by the target object, and controls the projection display device to perform the operation corresponding to the number of fingers according to the preset target operation mapping relationship.

[0149] The above embodiments describe a method for determining whether a target region meets the conditions for gesture recognition based on the head region, and then determining whether to perform gesture recognition on the hand region. By detecting the pixel area of ​​the head contour in the head region of the target depth image, it is inferred whether the head of the target object is facing the TOF sensor (projection display device), so as to avoid the projection display device responding to the user's unintentional hand movements.

[0150] The above embodiments describe a method for determining whether a target region meets the conditions for gesture recognition based on the hand and torso regions, and then determining whether to perform gesture recognition on the hand region. This is achieved by detecting the depth relationship between the hand and torso regions in the target depth image to determine whether the main control circuit should perform gesture recognition on the hand region. It should be noted that... Figure 7 , Figure 8 as well as Figure 9 The corresponding gesture control methods include three gesture recognition conditions: the target area includes the head area, or the head area and the hand area, or the torso area and the hand area. These conditions can be combined according to the user's sensitivity requirements for misidentification detection.

[0151] The following embodiments further illustrate how the main control circuit performs gesture recognition based on the positional relationship between the hand region and the head region in the target depth image.

[0152] Please see Figure 11 , Figure 11 This is a flowchart illustrating the determination of a target operation mapping relationship in the gesture control method disclosed in this application, which may include the following steps:

[0153] Step 1101: When the main control circuit determines that the gesture recognition function of the projection display device is enabled, it performs segmentation processing on the target depth image to obtain the hand region and head region in the target depth image.

[0154] Step 1102: If the hand region and head region meet the gesture recognition conditions, the main control circuit determines the target position relationship between the head region and the hand region in the target depth image, and determines the target operation mapping relationship as the preset operation mapping relationship corresponding to the target position relationship among multiple preset operation mapping relationships.

[0155] In some possible embodiments, the projection display device includes multiple preset operation mapping relationships, different preset operation mapping relationships corresponding to different preset positional relationships between the head region and the hand region, and the projection display device further includes:

[0156] The main control circuit determines the target position relationship between the head region and the hand region in the target depth image, and determines the target operation mapping relationship as the preset operation mapping relationship that corresponds to the target position relationship among multiple preset operation mapping relationships.

[0157] It should be noted that, in addition to being determined by the current projection content of the projection display device, in some embodiments, the main control circuit can also determine the target operation mapping relationship based on the target position relationship between the head region and the hand region in the target depth image.

[0158] In some possible embodiments, the target operation mapping relationship can be determined by combining the current projected content of the projection display device and the target positions of the head region and hand region in the target depth image.

[0159] For example, in some embodiments, after determining that the current projected content of the projection display device is video content, the target operation mapping relationship can be determined based on the target position relationship between the head region and the hand region in the target depth image. This is the target operation mapping relationship corresponding to the target position relationship among multiple preset operation mapping relationships existing in the scenario where the projected content is video content. In this way, a more flexible and diverse gesture control experience can be achieved.

[0160] In some possible embodiments, the target positional relationship between the head region and the hand region in the target depth image can be a depth-related positional relationship between the head region and the hand region in the target depth image. For example, the target positional relationship can be determined by comparing the regional depth values ​​of the head region and the hand region in the target depth image. The regional depth value can be the depth value of the center pixel of the region or the average depth value of all pixels in the region. In this case, multiple preset operation mapping relationships can include preset operation mapping relationships corresponding to the regional depth value of the head region being greater than or equal to the regional depth value of the hand region, and preset operation mapping relationships corresponding to the regional depth value of the head region being less than the regional depth value of the hand region. In addition, when the positional relationship is depth-related, the target operation mapping relationship corresponding to the target positional relationship can also be determined by calculating the difference between the regional depth values ​​of the head region and the hand region in the target depth image. For example, when the difference is less than 30cm, the target operation mapping relationship is determined to be one of multiple preset operation mapping relationships; when the difference is greater than or equal to 30cm, the target operation mapping relationship is determined to be another of multiple preset operation mapping relationships. Multiple zone levels can be set according to actual needs to achieve a more flexible gesture control strategy.

[0161] In some possible embodiments, the target positional relationship includes the magnitude relationship between the target pixel distance and a preset distance threshold. The target pixel distance includes the pixel distance between the center pixel of the head region and the center pixel of the hand region. The preset distance threshold includes at least one distance value, which is determined based on the depth value of the center pixel of the head region and / or the depth value of the center pixel of the hand region.

[0162] It should be noted that some users are more sensitive to the distance between their hands and head than to depth. Therefore, the target position relationship can also be set as the relationship between the target pixel distance between the head area and the hand area and a preset distance threshold. In this way, users can achieve diverse control of the projection display device simply by adjusting the relative distance between their hands and head (i.e., moving their hands closer or further away), even when extending the same number of fingers, based on the different distances between their hands and head.

[0163] It should be noted that the target pixel distance between the head region and the hand region can be the pixel distance between the center pixel of the head region and the center pixel of the hand region, or it can be the maximum or minimum pixel distance value that can be obtained by connecting any pixel in the head region and any pixel in the hand region. No limitation is made here.

[0164] Understandably, the distance between the target object and the TOF sensor directly affects the target pixel distance between the hand region and the head region in the target depth image. Therefore, a preset distance threshold can be determined based on the depth value of the center pixel of the head region and / or the depth value of the center pixel of the hand region (for example, selecting one of the two depth values, or calculating the average of the two depth values ​​and multiplying it by a preset coefficient to obtain the preset distance threshold) to determine the target positional relationship between the head region and the hand region in the target depth image.

[0165] It should be noted that the preset distance threshold includes at least one distance value. The more distance values ​​included in the preset distance threshold, the more controllable data can be applied when the target object extends the same number of fingers.

[0166] For example, if the preset distance threshold is determined based on the depth value of the center pixel of the head region, and the depth value of the center pixel of the head region is determined to be 2m, and if the preset distance threshold includes a distance value of 30 (in pixels), when the target pixel distance between the head region and the hand region is less than 30, the target operation mapping relationship can be determined as the operation mapping relationship corresponding to a target pixel distance less than 30. Conversely, when the target pixel distance between the head region and the hand region is greater than or equal to 30, the target operation mapping relationship can be determined as the operation mapping relationship corresponding to a target pixel distance greater than or equal to 30. If the preset distance threshold includes two distance values, 15 and 30 (in pixels), then when the target pixel distance is less than 15, greater than or equal to 15 and less than 30, or greater than or equal to 30, the main control circuit will obtain different target operation mapping relationships. While this provides the target object with richer control options, at the same time, the target object needs to more carefully control the distance between the hand and head to avoid the projection display device recognizing the target object's gestures as too large or too small, resulting in a response contrary to the target object's intention.

[0167] In some possible embodiments, the preset distance threshold includes a second distance threshold, and the main control circuit determines the target operation mapping relationship as the preset operation mapping relationship corresponding to the target position relationship among multiple preset operation mapping relationships, including:

[0168] The main control circuit determines the target operation mapping relationship as the preset operation mapping relationship corresponding to the case where the distance to the target pixel is less than the second distance threshold among multiple preset operation mapping relationships, or the main control circuit determines the target operation mapping relationship as the preset operation mapping relationship corresponding to the case where the distance to the target pixel is greater than or equal to the second distance threshold among multiple preset operation mapping relationships.

[0169] By setting a preset distance threshold that includes only the second distance threshold, the control logic can be greatly simplified, reducing the complexity of user operations. In this case, the user only needs to focus on whether the relative distance between the hand and head exceeds or does not reach this second distance threshold to easily trigger the corresponding operation mapping relationship.

[0170] In some possible embodiments, a relatively large preset coefficient can be introduced when determining the preset distance threshold (second distance threshold) based on the depth value of the center pixel of the head region and / or the depth value of the center pixel of the hand region. This aims to ensure that the main control circuit only recognizes the target object as belonging to one of the two preset operation mapping relationships defined by the second distance threshold when the target object clearly extends its hand, making the distance between the hand and the head reach a relatively large distance. This effectively reduces erroneous operation of the projection display device caused by improper distance control by the target object, thereby improving the user experience.

[0171] In some embodiments, when the main control circuit determines that the target pixel distance is less than the second distance threshold, the target operation mapping relationship can be as shown in Table 1 above. When the main control circuit determines that the target pixel distance is greater than or equal to the second distance threshold, the target operation mapping relationship can be as shown in Table 2 above.

[0172] Step 1103: The main control circuit performs gesture recognition on the hand area, obtains the number of fingers extended by the target object, and controls the projection display device to perform the operation corresponding to the number of fingers according to the target operation mapping relationship.

[0173] The above embodiments illustrate that the main control circuit can determine the target operation mapping relationship based on the positional relationship between the hand region and the head region in the target depth image. The target object can change the positional relationship between the hand and the head so that the target object can perform different control operations on the projection display device when extending the same number of fingers, thereby improving the flexibility of gesture control.

[0174] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0175] Based on the foregoing embodiments, this application provides a main control circuit. The device includes multiple modules and units included in each module, which can be implemented by a processor; of course, it can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP), or field programmable gate array (FPGA), etc.

[0176] Furthermore, for a clearer and more detailed description of the main control circuit disclosed in the above embodiments, please refer to [link to relevant documentation]. Figure 12 , Figure 12 This is a schematic diagram of the main control circuit disclosed in the embodiments of this application. Figure 12 The main control circuit shown includes a gesture recognition module 1201, an image processing module 1202, and a mis-recognition detection module 1203, wherein:

[0177] The gesture recognition module 1201, when it is determined that the gesture recognition function of the projection display device is enabled and the target area included in the target depth image acquired by the TOF sensor meets the gesture recognition conditions, performs gesture recognition on the hand area included in the target depth image, obtains the number of fingers extended by the target object, and controls the projection display device to perform the operation corresponding to the number of fingers according to the preset target operation mapping relationship.

[0178] The image processing module 1202 is used to segment the target depth image acquired by the TOF sensor to obtain the target region in the target depth image. The target region includes the head region, or the target region includes the head region and the hand region, or the target region includes the torso region and the hand region.

[0179] The false recognition detection module 1203 is used to determine whether the target region included in the target depth image meets the gesture recognition conditions.

[0180] In some possible embodiments, the target region includes a head region and a hand region. The misidentification detection module 1203 is further configured to obtain the pixel coordinates of the center pixel of the head region and the pixel coordinates of the center pixel of the hand region, calculate the pixel distance between the pixel coordinates of the center pixel of the head region and the pixel coordinates of the center pixel of the hand region, and determine a first distance threshold based on the depth value of the center pixel of the head region and / or the depth value of the center pixel of the hand region, and determine whether the hand region and the head region meet the gesture recognition conditions.

[0181] In some possible embodiments, the target region includes a head region. The misidentification detection module 1203 is further used to obtain the pixel area of ​​the head contour in the head region and the depth value of the center pixel of the head region; determine the area threshold according to the depth value of the center pixel of the head region and a preset area mapping relationship, the preset area mapping relationship including the mapping relationship between preset depth value and preset area, and determine whether the hand region meets the gesture recognition conditions.

[0182] In some possible embodiments, the target region includes a torso region and a hand region. The misidentification detection module 1203 is further used to determine whether the hand region is within the torso region; if it is determined that the hand region is within the torso region, the depth average of each pixel of the hand contour of the target object in the hand region and the depth average of each pixel in the hand region excluding the hand contour are obtained; and it is determined whether the hand region and the torso region meet the gesture recognition conditions.

[0183] In some possible embodiments, before performing gesture recognition on the hand region included in the target depth image, the gesture recognition module 1201 is further configured to acquire depth information corresponding to the hand region, determine a recognition plane based on the depth information, the depth information including the maximum or minimum depth value corresponding to the hand contour of the target object in the hand region; if it is determined that there is a depth value in each pixel of the hand contour whose depth difference with the recognition plane is greater than a preset error threshold, the projection display device is controlled to output prompt information, the prompt information being used to prompt the target object to adjust its hand posture.

[0184] In some possible embodiments, the projection display device includes multiple preset operation mapping relationships, and different preset operation mapping relationships correspond to different preset positional relationships between the head region and the hand region. The gesture recognition module 1201 is also used to determine the target positional relationship between the head region and the hand region in the target depth image, and to determine the target operation mapping relationship as the preset operation mapping relationship that corresponds to the target positional relationship among the multiple preset operation mapping relationships.

[0185] In some possible embodiments, the target position relationship includes the magnitude relationship between the target pixel distance and a preset distance threshold. The target pixel distance includes the pixel distance between the center pixel of the head region and the center pixel of the hand region. The preset distance threshold includes at least one distance value. The preset distance threshold is determined based on the depth value of the center pixel of the head region and / or the depth value of the center pixel of the hand region. The preset distance threshold includes a second distance threshold. The gesture recognition module 1201 is further used to determine that the target operation mapping relationship is the preset operation mapping relationship corresponding to the case where the target pixel distance is less than the second distance threshold among a plurality of preset operation mapping relationships. Alternatively, the main control circuit determines that the target operation mapping relationship is the preset operation mapping relationship corresponding to the case where the target pixel distance is greater than or equal to the second distance threshold among a plurality of preset operation mapping relationships.

[0186] In some possible embodiments, the main control circuit also includes a projection screen control module, which is used to determine the target screen area in the projection screen of the projection display device based on the pixel coordinates of each pixel in the head area, and reduce the pixel brightness of the target screen area.

[0187] In some possible embodiments, the projection display device includes multiple preset operation mapping relationships. Different preset operation mapping relationships correspond to different preset projection content of the projection display device. The projection content includes video content or non-video content. The gesture recognition module 1201 is also used to obtain the current projection content of the projection display device and determine the target operation mapping relationship as the preset operation mapping relationship that corresponds to the current projection content among the multiple preset operation mapping relationships.

[0188] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0189] It should be noted that, in the embodiments of this application... Figure 12The module division of the main control circuit shown is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical units, or have two or more units integrated into one unit. The integrated units can be implemented in hardware or as software functional units, or a combination of software and hardware. In the embodiments of this application, if the above method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0190] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A projection display device, characterized in that, The projection display device includes: A projection module is used to project an optical signal corresponding to a projection signal and display an image or video corresponding to the projection signal, wherein the projection signal is an image signal or a video signal. A time-of-flight (TOF) sensor is used to acquire depth images. The TOF sensor is disposed on the side of the projection module in the projection display device that projects the light signal. The acquisition area of ​​the TOF sensor and the projection area of ​​the projection module overlap. The main control circuit is used to acquire the depth image collected by the TOF sensor, perform gesture recognition on the depth image collected by the TOF sensor, and control the projection display device to perform the operation corresponding to the gesture recognition result. When the main control circuit determines that the gesture recognition function of the projection display device is enabled and the target area included in the target depth image acquired by the TOF sensor meets the gesture recognition conditions, it performs gesture recognition on the hand area included in the target depth image, obtains the number of outstretched fingers of the target object, and controls the projection display device to execute the operation corresponding to the number of fingers according to a preset target operation mapping relationship. The target area includes a head area, or the target area includes the head area and the hand area, or the target area includes a torso area and the hand area; the target operation mapping relationship includes the mapping relationship between a preset number of fingers and a preset operation of the projection display device, and the gesture recognition result includes the number of fingers extended by the object.

2. The projection display device according to claim 1, characterized in that, The target area includes the head area and the hand area. The gesture recognition condition includes that the pixel distance between the pixel coordinates of the center pixel of the head area and the pixel coordinates of the center pixel of the hand area is less than a first distance threshold. The first distance threshold is determined based on the depth value of the center pixel of the head area and / or the depth value of the center pixel of the hand area.

3. The projection display device according to claim 1, characterized in that, The target region includes the head region, and the gesture recognition condition includes that the pixel area of ​​the head contour is greater than or equal to an area threshold. The head contour is obtained by the main control circuit based on the head region, and the area threshold is determined based on the depth value of the center pixel of the head region and a preset area mapping relationship. The area mapping relationship includes a preset depth value and a preset area mapping relationship.

4. The projection display device according to claim 1, characterized in that, The target area includes the torso area and the hand area. The gesture recognition conditions include that the hand area is located within the torso area, and the difference between the average depth of each pixel in the hand contour and the average depth of each pixel outside the hand contour is greater than or equal to a preset depth threshold. The hand contour is obtained by the main control circuit based on the hand area.

5. The projection display device according to any one of claims 1-4, characterized in that, Before the main control circuit performs gesture recognition on the hand region included in the target depth image, the projection display device further includes: The main control circuit acquires the depth information corresponding to the hand region, and determines the recognition plane based on the depth information. The depth information includes the maximum or minimum depth value corresponding to the hand contour of the target object in the hand region. When the main control circuit determines that the depth difference between the depth value of each pixel point of the hand contour and the depth of the recognition plane is greater than a preset error threshold, it controls the projection display device to output prompt information, which is used to prompt the target object to adjust its hand posture.

6. The projection display device according to any one of claims 1-4, characterized in that, The projection display device includes multiple preset operation mapping relationships, with different preset operation mapping relationships corresponding to different preset positional relationships between the head region and the hand region. The projection display device also includes: The main control circuit determines the target position relationship between the head region and the hand region in the target depth image, and determines the target operation mapping relationship as the preset operation mapping relationship that corresponds to the target position relationship among the plurality of preset operation mapping relationships.

7. The projection display device according to claim 6, characterized in that, The target position relationship includes the magnitude relationship between the target pixel distance and the preset distance threshold. The target pixel distance includes the pixel distance between the center pixel of the head region and the center pixel of the hand region. The preset distance threshold includes at least one distance value. The preset distance threshold is determined based on the depth value of the center pixel of the head region and / or the depth value of the center pixel of the hand region.

8. The projection display device according to claim 7, characterized in that, The preset distance threshold includes a second distance threshold. The main control circuit determines the target operation mapping relationship as the preset operation mapping relationship that corresponds to the target position relationship among the plurality of preset operation mapping relationships, including: The main control circuit determines that the target operation mapping relationship is a preset operation mapping relationship among the plurality of preset operation mapping relationships that corresponds to the case where the distance to the target pixel is less than the second distance threshold, or the main control circuit determines that the target operation mapping relationship is a preset operation mapping relationship among the plurality of preset operation mapping relationships that corresponds to the case where the distance to the target pixel is greater than or equal to the second distance threshold.

9. The projection display device according to claim 1, characterized in that, The projection display device further includes: The main control circuit determines the target image area in the projected image of the projection display device based on the pixel coordinates of each pixel in the head region, and reduces the pixel brightness of the target image area.

10. The projection display device according to claim 1, characterized in that, The projection display device includes multiple preset operation mapping relationships. Different preset operation mapping relationships correspond to different preset projection content of the projection display device. The projection content includes video content or non-video content. The projection display device also includes: The main control circuit acquires the current projected content of the projection display device and determines the target operation mapping relationship as the preset operation mapping relationship that corresponds to the current projected content among the multiple preset operation mapping relationships.

Images