Projector gesture control method, system and device, medium and program product
By setting masks in the projection device to enhance contrast, the problem of targets being blocked and color space features in the projection device is solved, and accurate recognition and detection of gestures is achieved, improving the recognition accuracy and user experience of the projector.
Patent Information
- Application Number
- CN202411209173.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2025-01-07
AI Technical Summary
In projection equipment, since the camera and optical machine are embedded in the same position, the light source of the projection screen will block the target, causing the target to be blocked or the color space features to be broken, reducing the accuracy of target detection, especially when you need to see the projection screen clearly or perform gesture control , The traditional method of using whiteboard diagrams is not applicable, especially for small targets and night or when the lights are turned off.
By setting the mask, the contrast between the projection area and the target user is enhanced, the gesture image is obtained and parsed to execute control instructions, and instead of the traditional whiteboard image operation method, the mask is a white image with transparency between the preset area values, superimposed on the projection Area to reduce interface brightness and improve contrast.
It achieves the accuracy of gesture recognition without affecting the viewing of the projection area, and reduces the error detection rate of small targets, improves the accuracy of projector gesture detection and recognition, and improves the user experience.
Smart Images
Figure CN119271041A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of projection gesture control, and in particular to a projector gesture control method, system, device, medium and program product. Background Art
[0002] In the projection device, since the camera and the optical machine are embedded in the same position of the device, the light source of the projection picture comes from the projection of the optical machine, and the picture will be projected onto all surfaces of objects in the projection area. When the camera collects the interactive or manipulated image of the target, part of the picture projected by the optical machine will exist on the surface of the objects in the area, which will naturally block the target. In addition, the color space of the projection picture is relatively complex. When the color of the projection area picture is consistent with or close to the color of the target itself, the target will be blocked or split into different color space features. The image processing AI algorithms developed and deployed in the projection device will have reduced accuracy when identifying targets.
[0003] In the related technology, some algorithms involving target detection deployed on projection devices use a whiteboard method to deal with this problem, that is, an opaque white image is projected on the target through an optical machine so that the camera can see the target in front clearly. However, the above-mentioned whiteboard method has limitations. For some functions that require a clear view of the projected image, or when the user needs to interact with the AI algorithm based on the projected image, such as dragging and clicking controlled by projection gestures, the user needs to see the projected image clearly, and the algorithm also needs to accurately identify the user's gesture instructions and make corresponding operations. The whiteboard method will no longer be applicable.
[0004] If a whiteboard image is not used, partial coverage of the projection screen will have little effect on the detection of large targets with obvious features, but it will have a greater impact on small targets, such as hands and faces, and the impact will be greater at night or when the lights are off. Finally, the images of objects or people in the projection screen will also interfere with recognition, causing the people or hand movements in the projection screen to be mistakenly detected as real user commands. Summary of the invention
[0005] The present application provides a projector gesture control method, system, device, medium and program product to at least solve the problem that the related art determines that the projector gesture control scheme has limitations, resulting in poor gesture recognition effect.
[0006] In a first aspect, the present application provides a projector gesture control method, the method comprising:
[0007] In response to a target user in the projection area, setting a mask, the mask being used to enhance the contrast between the interface of the projection area and the target user;
[0008] Acquire a gesture image of the target user in front of the mask area, recognize the gesture image and parse and acquire a control instruction;
[0009] Perform gesture control on the interface of the projection area according to the control instruction.
[0010] Optionally, the step of setting a mask in response to a target user in the projection area includes:
[0011] Acquire the target user image in the projection area within a first preset time;
[0012] Locally superimposing the mask on the projection area where the target user image is located to form a first mask area;
[0013] The mask is a white image whose transparency is between preset area values, and is used to reduce the interface brightness of the first mask area.
[0014] Optionally, acquiring a gesture image of the target user in front of the mask area, recognizing the gesture image and parsing and acquiring a control instruction includes:
[0015] A first gesture image of the target user in front of the first mask area is obtained within a second preset time, the first gesture image is recognized and parsed to obtain a first control instruction; wherein the first control instruction is used to expand the mask area to form a second mask area.
[0016] Optionally, after identifying the first gesture image and parsing and acquiring the first control instruction, the method further includes:
[0017] A second gesture image of the target user in front of the second mask area is obtained within a third preset time, the second gesture image is identified and parsed to obtain a second control instruction; wherein the second control instruction is used for interface gesture control of the second mask area.
[0018] Optionally, the area of the second mask region globally covers the interface of the projection region.
[0019] Optionally, before responding to the target user in the projection area, the method further includes:
[0020] Receive a trigger instruction from the gesture switch to turn on the gesture control mode of the projection area.
[0021] Optionally, performing gesture control on the interface of the projection area according to the control instruction, then includes:
[0022] If the second gesture image is not acquired within a third preset time, the mask is exited.
[0023] In a second aspect, the present application provides a projector gesture control system, the system comprising:
[0024] A mask setting module, used to set a mask in response to a target user in the projection area, wherein the mask is used to enhance the contrast between the interface of the projection area and the target user;
[0025] An instruction parsing module, used to obtain a gesture image of the target user in front of the mask area, identify the gesture image and parse it to obtain a control instruction;
[0026] The gesture control module is used to perform gesture control on the interface of the projection area according to the control instruction.
[0027] In a third aspect, the present application provides an electronic device, including:
[0028] Processor; and
[0029] A memory, configured to store executable instructions of the processor;
[0030] Wherein, the processor is configured to execute the projector gesture control method of the first aspect by executing the executable instructions.
[0031] In a fourth aspect, the present application provides a computer-readable storage medium having a program stored thereon, and when the program is executed by a processor, the projector gesture control method of the first aspect is implemented.
[0032] Compared with the related art, the projector gesture control method, system, device, medium and program product provided by the present application have at least the following technical effects:
[0033] By masking and superimposing the projection area interface during gesture interaction, the traditional whiteboard operation method is replaced, and the contrast between the target user and the projection area interface is enhanced during gesture recognition. While not affecting the visibility of the projection area interface, the target user's gesture is accurately recognized, with little impact on user operation. After the image features are weakened, the probability of the target user or the target user's gesture being misdetected as a real user command is greatly reduced, which greatly improves the detection accuracy of the projection algorithm for small target objects, thereby improving the accuracy of projector gesture detection and recognition without affecting user operation and improving user experience.
[0034] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0036] Figure 1 The figure is a flow chart of a projector gesture control method according to an exemplary embodiment.
[0037] Figure 2 is a flowchart of step S101 according to an exemplary embodiment.
[0038] Figure 3A to Figure 3C is a comparison diagram of a projection area and a first mask area according to an exemplary embodiment.
[0039] Figure 4 is a flowchart of step S102 according to an exemplary embodiment.
[0040] Figure 5 is a flow chart of a projector gesture control method according to another exemplary embodiment.
[0041] Figure 6 is a block diagram of a projector gesture control system according to an exemplary embodiment.
[0042] Figure 7 The figure is a flow chart of a projector gesture control system according to an exemplary embodiment. DETAILED DESCRIPTION
[0043] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.
[0044] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.
[0045] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.
[0046] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.
[0047] In related technologies, some algorithms involving target detection deployed on projection devices are used to deal with the problem that the accuracy of the image processing AI algorithms developed and deployed in the projection devices is reduced when identifying targets because the targets are blocked or split into different color space features. The whiteboard method is used, that is, an opaque white image is projected through an optical machine and projected on the target so that the camera can see the target in front clearly. However, the above-mentioned whiteboard method has limitations. For some functions that require a clear view of the projected image, or when the user needs to interact with the AI algorithm based on the projected image, such as dragging and clicking controlled by projection gestures, the user needs to see the projected image clearly, and the algorithm also needs to accurately identify the user's gesture instructions and make corresponding operations. The whiteboard method will no longer be applicable.
[0048] If a whiteboard image is not used, partial coverage of the projection screen will have little effect on the detection of large targets with obvious features, but it will have a greater impact on small targets, such as hands and faces, and the impact will be greater at night or when the lights are off. Finally, the images of objects or people in the projection screen will also interfere with recognition, causing the people or hand movements in the projection screen to be mistakenly detected as real user commands.
[0049] Based on the above situation, the embodiments of the present invention provide a projector gesture control method, system, device, medium and program product, which are described in detail below in conjunction with specific embodiments and drawings.
[0050] Example 1
[0051] An embodiment of the present invention provides a gesture control method for a projector. Figure 1 FIG. 1 is a flow chart of a projector gesture control method according to an exemplary embodiment. Figure 1 As shown, the method includes:
[0052] S101, in response to a target user in a projection area, setting a mask, where the mask is used to enhance the contrast between the interface of the projection area and the target user;
[0053] In this embodiment, Figure 2 FIG. 1 is a flowchart of step S101 according to an exemplary embodiment. Figure 2 Specifically, in response to a target user in the projection area, setting a mask includes:
[0054] S1011, acquiring an image of a target user within a projection area within a first preset time;
[0055] S1012, partially superimposing a mask on the projection area where the target user image is located to form a first mask area; wherein the mask is a white image with a transparency between preset area values, and is used to reduce the interface brightness of the first mask area;
[0056] In this embodiment, Figure 3A to Figure 3C FIG. 1 is a comparison diagram of a projection area and a first mask area according to an exemplary embodiment. Figure 3A to Figure 3C, because the camera and the optical machine are embedded in the same position of the projection device, the light source of the projection picture comes from the optical machine projection, and the picture will be projected onto all the surfaces of objects in the projection area. The camera is used to collect the interaction or manipulation images of the target user; when the camera collects the interaction or manipulation images of the target, part of the picture projected by the optical machine will exist on the surface of the objects in the area, which naturally blocks the target; and the color space of the projection picture is relatively complex. When the color of the projection area picture is consistent with or close to the color of the target itself, the target will be blocked or split into different color space features, and the image processing AI algorithms developed and deployed in the projection device will have reduced accuracy in identifying targets; in order to deal with this problem, some algorithms involving target detection on the deployment projection device adopt the method of whiteboard drawing, that is, an opaque white image is projected by the optical machine onto the target so that the camera can see the target in front clearly. However, the above-mentioned whiteboard drawing solution has limitations. For some functions that require a clear view of the projected image, or that involve users interacting with the AI algorithm based on the projected image, such as dragging and clicking controlled by projection gestures, the user needs to see the projected image clearly, and the algorithm also needs to accurately identify the user's gesture instructions and make corresponding operations. The whiteboard drawing method will no longer be applicable.
[0057] Furthermore, in this embodiment, the mask is a white image with a transparency between the preset area values. We judge the effect example by two levels: gesture detection accuracy and whether the operation screen can be seen clearly;
[0058] For gesture detection accuracy, the yolov8 target detection model is used to identify the coordinates and categories of the target user's hand. The coordinate accuracy of the target user's hand is the IOU overlap of the predicted value and the calibration value box. If it exceeds 90%, the detection is considered accurate. The category accuracy of the target user's hand is the complete match of the category judgment. It is calculated according to the following formula (1):
[0059] Gesture detection accuracy = coordinate accuracy * category accuracy (1)
[0060] Whether the operation screen can be seen clearly is based on the manual judgment method of whether the target user can see it clearly;
[0061] The evaluation data comes from 3000 test data collected in the laboratory, including a total of 4 gesture images including the first gesture image and the second gesture image. The results are shown in Table 1:
[0062]
[0063] Table 1
[0064] In this embodiment, the preset area value of the mask transparency is 60%-80%, which has a good gesture detection accuracy and a clear operation screen effect. When the mask is preferably a white image with a transparency of 70%, a semi-transparent mask is superimposed on the screen of the projection area, which can effectively reduce the interface brightness of the first mask area and the subsequent second mask area, forming a high contrast difference with the target user, so that the operation screen can be clearly seen and the gesture detection accuracy is better, and gesture recognition is not affected.
[0065] Therefore, in the technical solution of the above-mentioned embodiment, a method of using a mask to superimpose the projection area interface is proposed to replace the traditional whiteboard operation method. The picture features of the projection area interface are weakened during gesture recognition, but the transparency control of the mask does not affect the visibility of the projection area interface, thereby achieving accurate recognition of the target user's gesture, with little impact on user operation. After the picture features are weakened, the probability of the target user or the target user's gesture being misdetected as a real user command will be greatly reduced, and the detection accuracy of the projection algorithm for small target objects will be greatly improved, thereby achieving the improvement of the projector gesture detection and recognition accuracy without affecting user operation, thereby improving the user experience.
[0066] S102, obtaining a gesture image of the target user in front of the mask area, recognizing the gesture image and parsing it to obtain a control instruction;
[0067] In this embodiment, Figure 4 FIG. 1 is a flowchart of step S102 according to an exemplary embodiment. Figure 4 Specifically, the gesture image of the target user in front of the mask area is obtained, the gesture image is recognized and parsed to obtain the control instruction, including:
[0068] S1021. Acquire a first gesture image of the target user in front of the first mask area within a second preset time, identify the first gesture image and parse to obtain a first control instruction; wherein the first control instruction is used to expand the mask area to form a second mask area; further, in this embodiment, after the area of the second mask area is expanded, the interface of the projection area is globally covered.
[0069] S1022. Obtain a second gesture image of the target user in front of the second mask area within a third preset time, identify the second gesture image and parse it to obtain a second control instruction; wherein the second control instruction is used for interface gesture control of the second mask area.
[0070] In this embodiment, specifically, the interactive operation of the interface is determined according to the second control instructions obtained by parsing different second gesture images according to the pre-stored second control instruction mapping relationship, wherein it can be understood that the second control instructions obtained by parsing can be interactive operations such as expanding the button area, dragging the screen, moving the focus, clicking to confirm, etc.
[0071] S103: Perform gesture control on the interface of the projection area according to the control instruction.
[0072] Optionally, Figure 5 FIG. 1 is a flow chart of a projector gesture control method according to another exemplary embodiment. Figure 5 , before step S101, it also includes:
[0073] S100, receiving a trigger instruction of a gesture switch to turn on a gesture control mode of a projection area; before gesture recognition and control, it is necessary to first determine whether to turn on the projection device to enter the gesture control mode.
[0074] Optionally, Figure 5 FIG. 1 is a flow chart of a projector gesture control method according to another exemplary embodiment. Figure 5 , after step S103, including:
[0075] S104, if the second gesture image is not obtained within the third preset time, the mask is exited; it can be understood that if the camera does not obtain the second gesture image of the target user within a certain period of time after the gesture control is completed, the gesture control mode is exited by default.
[0076] In summary, Figure 7 FIG. 1 is a flow chart of a projector gesture control system according to an exemplary embodiment. Figure 5 and 7 The projector gesture control method provided by the embodiment of the present invention adopts a method of masking and superimposing the projection area interface to replace the traditional whiteboard operation method, and enhances the contrast between the target user and the projection area interface during gesture recognition, while not affecting the visibility of the projection area interface, and accurately recognizes the target user's gesture, with little impact on user operation. After the image features are weakened, the probability of the target user or the target user's gesture being misdetected as a real user command will be greatly reduced, and the detection accuracy of the projection algorithm for small target objects will be greatly improved, thereby achieving the improvement of the accuracy of projector gesture detection and recognition without affecting user operation, and improving user experience.
[0077] Example 2
[0078] Embodiment 2 of the present invention provides a projector gesture control system. Figure 6 FIG. 1 is a block diagram of a projector gesture control system according to an exemplary embodiment. Figure 6 , the system comprises:
[0079] A mask setting module 201, used to set a mask in response to a target user in the projection area, the mask being used to enhance the contrast between the interface of the projection area and the target user;
[0080] The instruction parsing module 202 is used to obtain the gesture image of the target user in front of the mask area, recognize the gesture image and parse it to obtain the control instruction;
[0081] The gesture control module 203 is used to perform gesture control on the interface of the projection area according to the control instruction.
[0082] The projector gesture control system provided in Example 2 of the present invention adopts a method of masking and superimposing the projection area interface to replace the traditional whiteboard image operation method, and enhances the image contrast between the target user and the projection area interface during gesture recognition. It does not affect the visibility of the projection area interface while achieving accurate recognition of the target user's gesture, with little impact on user operation. After the image features are weakened, the probability of the target user or the target user's gesture being misdetected as a real user command will be greatly reduced, and the detection accuracy of the projection algorithm for small target objects will be greatly improved, thereby achieving the improvement of the projector gesture detection and recognition accuracy without affecting the user's operation and improving the user experience.
[0083] Example 3
[0084] Embodiment 3 of the present invention provides an electronic device, including:
[0085] Processor; and
[0086] A memory for storing executable instructions of the processor;
[0087] The processor is configured to execute the projector gesture control method of the above-mentioned embodiment 1 by executing executable instructions.
[0088] Example 4
[0089] Embodiment 4 of the present invention provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the projector gesture control method of the above embodiment 1 is implemented.
[0090] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.
[0091] In a possible implementation manner, the present invention may also be a computer program product, including: a computer program or instructions, and when the computer program or instructions are executed by a processor, the projector gesture control method of the above-mentioned embodiment 1 is implemented.
[0092] The program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or entirely on a remote device.
[0093] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0094] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A projector gesture control method, characterized in that: The method comprises: In response to a target user in the projection area, setting a mask, the mask being used to enhance the contrast between the interface of the projection area and the target user; Acquire a gesture image of the target user in front of the mask area, recognize the gesture image and parse and acquire a control instruction; Perform gesture control on the interface of the projection area according to the control instruction.
2. The projector gesture control method according to claim 1, characterized in that: The step of setting a mask in response to a target user in the projection area includes: Acquire the target user image in the projection area within a first preset time; Locally superimposing the mask on the projection area where the target user image is located to form a first mask area; The mask is a white image whose transparency is between preset area values, and is used to reduce the interface brightness of the first mask area.
3. The projector gesture control method according to claim 2, characterized in that: The step of obtaining a gesture image of the target user in front of the mask area, recognizing the gesture image and parsing and obtaining a control instruction includes: A first gesture image of the target user in front of the first mask area is obtained within a second preset time, the first gesture image is recognized and parsed to obtain a first control instruction; wherein the first control instruction is used to expand the mask area to form a second mask area.
4. The projector gesture control method according to claim 3, characterized in that: After the first gesture image is identified and the first control instruction is parsed and obtained, the method includes: A second gesture image of the target user in front of the second mask area is obtained within a third preset time, the second gesture image is identified and parsed to obtain a second control instruction; wherein the second control instruction is used for interface gesture control of the second mask area.
5. The projector gesture control method according to claim 3, characterized in that: The area of the second mask region globally covers the interface of the projection region.
6. The projector gesture control method according to claim 1, characterized in that: Before responding to the target user in the projection area, the method further includes: Receive a trigger instruction from the gesture switch to turn on the gesture control mode of the projection area.
7. The projector gesture control method according to claim 1, characterized in that: The step of performing gesture control on the interface of the projection area according to the control instruction comprises: If the second gesture image is not acquired within a third preset time, the mask is exited.
8. A projector gesture control system, characterized in that: The system comprises: A mask setting module, used to set a mask in response to a target user in the projection area, wherein the mask is used to enhance the contrast between the interface of the projection area and the target user; An instruction parsing module, used to obtain a gesture image of the target user in front of the mask area, identify the gesture image and parse it to obtain a control instruction; The gesture control module is used to perform gesture control on the interface of the projection area according to the control instruction.
9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the projector gesture control method according to any one of claims 1 to 7 by executing the executable instructions.
10. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the projector gesture control method described in any one of claims 1 to 7 is implemented.