Control method, control system, projection equipment and storage medium
By using inertial sensors and dynamically controlling infrared emitters in the stylus pen, the limitations of existing terminal devices in the pen tip and cap operation switching are solved, and convenient user interaction and high-reliability infrared interaction are achieved.
Patent Information
- Application Number
- CN202411867552.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-17
- Publication Date
- 2025-05-23
AI Technical Summary
Existing terminal devices that combine micro projectors and infrared control functions have limitations in interactive operation, especially when switching between pen tip operation and pen cap operation, relying on physical or virtual buttons, resulting in inconvenient operation and high cost.
By using an inertial sensor in the stylus to determine the operating state, and dynamically control the working state of the infrared transmitter according to the state, and sending corresponding control signals to the projection device to realize automatic identification and switching of the tip and cap operations.
The free switching control mode is realized in any scenario, reducing dependence on physical and virtual buttons, improving the convenience of user interaction, and improving the reliability of infrared interaction of projection devices.
Smart Images
Figure CN120029493A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of terminal technology, and in particular to a control method, a control system, a projection device and a storage medium. Background Art
[0002] Existing terminal devices that combine micro projectors with infrared control functions still face some limitations in interactive operations. For example, in the scenario of using a stylus pen to interact with a micro projector, the stylus pen tip operation and pen cap operation usually correspond to different interactive strategies, so that terminal devices with projection functions can realize more interactive scenarios. Summary of the invention
[0003] In a first aspect, an embodiment of the present application provides a control method, including: The stylus determines the operation state of the stylus according to the inertial sensor data; In response to the operating state being a pen tip operating state, the first infrared emitter of the stylus pen is in an excitable state and generates a first control signal corresponding to the pen tip operating state; The projection device receives the first control signal sent by the stylus pen; In response to the first control signal, the plurality of second infrared emitters of the projection device are in a closed state; In response to the operating state being a pen cap operating state, the first infrared emitter of the stylus pen is in an unexcitable state and generates a second control signal corresponding to the pen cap operating state; The projection device receives the second control signal sent by the stylus; In response to the second control signal, the plurality of second infrared emitters of the projection device are in an emission state to form an infrared array, wherein in a direction perpendicular to a projection plane formed by the projection device, the infrared array overlaps the projection plane.
[0004] In some embodiments, it also includes: The stylus pen acquires pressure sensing data of a pressure sensor, and the pressure sensor is arranged at the tip of the stylus pen; In response to the pressure sensing data exceeding a threshold, the first infrared emitter of the stylus pen is in an activatable state.
[0005] In some embodiments, wherein the stylus determines the operating state of the stylus according to the inertial sensor data, further comprising: The stylus determines the posture data of the stylus according to the inertial sensor data; The stylus determines an operation state of the stylus according to the gesture data.
[0006] In some embodiments, it also includes: The projection device captures a first light spot formed by the reflection of infrared light from the first infrared emitter of the stylus; The projection device identifies the pen tip operation of the stylus pen according to the position information of the first light spot in the projection plane.
[0007] In some embodiments, it also includes: The projection device captures a second light spot formed by the reflection of infrared light in the infrared array; The projection device identifies the pen cap operation of the stylus pen according to the position information of the second light spot in the projection plane.
[0008] In a second aspect, an embodiment of the present application provides a control method, including: The projection device receives the first control signal sent by the stylus pen, wherein the first control signal is configured to indicate that the stylus pen is in a pen tip operation state; In response to the first control signal, the plurality of second infrared emitters of the projection device are in a closed state; The projection device receives the second control signal sent by the stylus pen, wherein the second control signal is configured to indicate that the stylus pen is in a pen cap operation state; In response to the second control signal, the plurality of second infrared emitters of the projection device are in an emission state to form an infrared array, wherein in a direction perpendicular to a projection plane formed by the projection device, the infrared array overlaps the projection plane.
[0009] In some embodiments, it also includes: The projection device captures a first light spot formed by the reflection of infrared light from the first infrared emitter of the stylus pen in the projection plane; The projection device identifies the pen tip operation of the stylus pen according to the position information of the first light spot in the projection plane.
[0010] In some embodiments, it also includes: The projection device captures a second light spot formed by the reflection of infrared light in the infrared array; The projection device identifies the pen cap operation of the stylus pen according to the position information of the second light spot in the projection plane.
[0011] In a third aspect, an embodiment of the present application provides a control system, including a stylus pen and a projection device, wherein the stylus pen is configured as follows: determining an operating state of the stylus according to inertial sensor data; In response to the operating state being a pen tip operating state, the first infrared emitter of the stylus pen is in an excitable state and generates a first control signal corresponding to the pen tip operating state; In response to the operating state being a pen cap operating state, the first infrared emitter of the stylus pen is in an unexcitable state and generates a second control signal corresponding to the pen cap operating state; The projection device is configured as follows: The projection device receives the first control signal sent by the stylus; In response to the first control signal, the plurality of second infrared emitters of the projection device are in a closed state; The projection device receives the second control signal sent by the stylus; In response to the second control signal, the plurality of second infrared emitters of the projection device are in an emission state to form an infrared array, wherein in a direction perpendicular to a projection plane formed by the projection device, the infrared array overlaps the projection plane.
[0012] In a fourth aspect, the present application provides a projection device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the methods described above is implemented.
[0013] In a fifth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the methods described above.
[0014] In a sixth aspect, the present application provides a computer program product, including a computer program, which implements any of the methods described above when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0016] Figure 1 One of the scenario schematic diagrams provided for an embodiment of the present application; Figure 2 A second schematic diagram of a scenario provided for an embodiment of the present application; Figure 3 A third schematic diagram of a scenario provided for an embodiment of the present application; Figure 4One of the flow charts of a control method provided in one embodiment of the present application; Figure 5 A schematic diagram of the structure of a stylus pen provided in one embodiment of the present application; Fig. 6A One of the schematic diagrams of a scene of pen tip operation provided by an embodiment of the present application; Figure 6B A second schematic diagram of a scene of pen tip operation provided in an embodiment of the present application; Fig. 7A One of the schematic diagrams of the pen cap operation scenario provided by one embodiment of the present application; Figure 7B A second schematic diagram of a pen cap operation scenario provided by an embodiment of the present application; Figure 8 The second flowchart of the control method provided for one embodiment of the present application. DETAILED DESCRIPTION
[0017] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0018] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.
[0019] The main working principle of the projector is that the light source emits light, such as three-color light, and through a series of optical lighting systems, the light of the light source is evenly irradiated onto the display chip, such as DMD. The display chip receives the light and realizes the brightness change of a single pixel by controlling the reflection angle of the mirror, and realizes the grayscale change of a single pixel through PWM modulation, thereby displaying the image. After that, the projection lens at the front end of the projection enlarges the image on the display chip and projects it onto the corresponding screen. The light source of the projector can be an LED light source or a laser light source.
[0020] Existing terminal devices that combine projectors with infrared control functions, such as learning machines, use micro-projection to present images. Figure 1 As shown, the projector 10 can project images or videos onto a larger screen 20, achieving a display effect similar to that of a short-throw projector, and can be used in a variety of scenarios such as home, office, and school, providing a visual experience similar to that of a large-screen TV or projection screen. Figure 2 and Figure 3 As shown, in terms of interaction, two modes are supported: hand touch and pen tip touch. Hand touch allows the user to directly use fingers to perform touch operations on the screen 120, such as sliding, clicking, etc., similar to the interaction mode of a smart phone or tablet computer. Pen tip touch provides more sophisticated operation possibilities. The user can use a special pen tip (such as a stylus) to write, draw or perform more precise selection operations on the screen 120.
[0021] Furthermore, the stylus can also provide pen cap operations to achieve interactions such as erasing, canceling, and rejecting. Currently, the switching between pen tip operations and pen cap operations is usually achieved based on physical buttons on the stylus or virtual buttons displayed on the micro-projection screen. The physical buttons on the stylus are usually small in size and not easy to touch accurately, which affects the convenience of operation to a certain extent, and the addition of physical buttons increases the cost of the device. When relying on virtual buttons for switching, since the virtual buttons are usually only visible on the main interface, once the user enters a specific application interface, the switching function will become invalid, limiting the user's flexible operation in all scenarios.
[0022] Moreover, whether the operation mode of the stylus is switched through the physical buttons on the stylus and / or the virtual buttons of the projection display, the user's interaction cost needs to be increased. Since the button operation is not related to the state where the stylus tip is facing the screen or the pen cap is facing the screen, the user may not know which operation state the stylus will be in when pressing the button. In addition, for micro-projection devices combined with infrared functions, they usually use active infrared interaction technology (i.e., the micro-projection device emits infrared light) to achieve interaction with the hand or stylus. However, its infrared emission function is continuously open, which makes it easy to cause touch failure and waste of power consumption when using a stylus with an infrared emission function for interaction.
[0023] To this end, the present application provides a control method, which uses an inertial sensor on the stylus to determine the operating state of the stylus, and controls the working state of the corresponding stylus and the infrared emitter of the projection device in each touch state, thereby realizing free switching of control modes in any scenario, reducing the dependence on physical buttons and virtual buttons, increasing the reliability of infrared interaction between the stylus and the projection device, and improving the convenience of user interaction.
[0024] The control method provided in the embodiment of the present application can be applied to a control system including a stylus pen and a projection device, the projection device can be a large projection device or a micro projection device, and the projection device can receive various instructions of the stylus pen on the projection screen in the form of infrared to achieve interactions such as writing, clicking, sliding, erasing, canceling, etc. The projection device can also have various other functional devices such as microphones, speakers, wireless communication chips, etc. to achieve more interactive functions with users.
[0025] Figure 4 This is one of the flow charts of the control method provided in the embodiment of the present application. Figure 4 As shown, a control method is provided, including the following steps: step 410, step 420, step 430, step 440, step 450, step 460 and step 470. The steps of the method flow are only a possible implementation of the present application.
[0026] Step 410: The stylus determines the operation state of the stylus according to the inertial sensor data; The stylus is equipped with an inertial sensor (such as an accelerometer, gyroscope, etc.) that can sense the motion state of the stylus, such as acceleration, angular velocity, etc. Through the inertial sensor data, the stylus can determine whether it is currently in the pen tip operation state or the pen cap operation state.
[0027] It can be understood that the pen tip operation state described herein refers to the pen tip of the stylus pen facing the screen of the projection device, for example, the pen tip and the screen present a state greater than 10° (usually an acute angle), and the first distance between the pen tip and the screen is less than the second distance between the pen cap and the screen. The pen cap operation state described herein refers to the pen cap of the stylus pen facing the screen of the projection device, for example, the pen cap and the screen present a state greater than 10° (usually an acute angle or a right angle), and the first distance between the pen tip and the screen is greater than the second distance between the pen cap and the screen.
[0028] Step 420: In response to the operating state being a pen tip operating state, the first infrared emitter of the stylus pen is in an excitable state and generates a first control signal corresponding to the pen tip operating state; Here, the excitable state means that the first infrared emitter inside the stylus pen is not activated and emits infrared light immediately, but needs to meet certain trigger conditions, such as when the pressure sensitivity of the pen tip is greater than the pressure sensitivity setting value, the first infrared emitter inside the stylus pen emits infrared light.
[0029] When the stylus pen is in the pen tip operation state, the first infrared emitter of the stylus pen (located inside the stylus pen, usually close to the pen tip) enters an excitable state. In the excitable state, the first infrared emitter inside the stylus pen emits infrared light.
[0030] It should be understood that in order to ensure that the infrared light can be emitted from the pen tip, one or more light outlet channels for the infrared light to be emitted are usually provided around the pen tip of the stylus, so that the infrared light can be emitted from the inside of the pen tip. In some examples, multiple light outlet holes are provided around the side of the stylus close to the pen tip, and the multiple light outlet holes can be connected to the first infrared emitter, so that the infrared light emitted by the first infrared emitter can be transmitted to the surroundings of the stylus through the multiple light outlet holes.
[0031] In addition, when the first infrared emitter inside the stylus pen enters the excitable state, the stylus pen also generates a first control signal corresponding to the pen tip operation state, and sends the first control signal to the projection device to indicate that the stylus pen is in the pen tip operation state.
[0032] In one example, the stylus pen and the projection device communicate wirelessly via Bluetooth. When the first infrared transmitter of the stylus pen is in an excitable state, the stylus pen transmits a specific Bluetooth signal to the projection device to indicate that the stylus pen is in a pen tip operation state. It is understood that the stylus pen and the projection device can also communicate wirelessly via WiFi, UWB or NFC, for example.
[0033] Step 430: The projection device receives the first control signal sent by the stylus pen; The projection device receives a first control signal sent by the stylus pen, and determines that the stylus pen is in a pen tip operation state by decoding the first control signal.
[0034] Step 440: In response to the first control signal, the plurality of second infrared emitters of the projection device are in a closed state; It should be understood that the projection device includes a plurality of second infrared emitters, and these second infrared emitters are usually arranged in a specific layout to form an infrared array on the projection plane of the projection device.
[0035] When the projection device receives the first control signal, the multiple second infrared emitters inside the projection device are controlled to be turned off to prevent the infrared arrays emitted by the multiple second infrared emitters from interfering with the infrared light emitted by the first infrared emitter of the stylus when the pen tip is in operation.
[0036] Step 450: In response to the operating state being a pen cap operating state, the first infrared emitter of the stylus pen is in an unexcitable state and generates a second control signal corresponding to the pen cap operating state; When the stylus pen is in the cap operation state, the first infrared emitter enters an unexcitable state (does not emit infrared light even if a driving signal is applied). At the same time, the stylus pen generates a second control signal to indicate that the stylus pen is in the cap operation state.
[0037] Step 460: the projection device receives the second control signal sent by the stylus; The projection device receives a second control signal sent by the stylus pen, and determines that the stylus pen is in a pen cap operation state by decoding the second control signal.
[0038] Step 470: In response to the second control signal, the plurality of second infrared emitters of the projection device are in an emitting state to form an infrared array, wherein in a direction perpendicular to a projection plane formed by the projection device, the infrared array overlaps with the projection plane.
[0039] When the projection device receives the second control signal, the plurality of second infrared emitters inside the projection device are controlled to turn on and emit infrared light to form an infrared array.
[0040] Here, the projection device can project a projection area on its projection plane, and the projection area is a content interface displayed on the projection plane. The projection plane can be a variety of flat surfaces, such as a desktop, a wall, a curtain, etc.
[0041] In order to realize the interactive function, the projection device has multiple second infrared emitters built in, which are arranged in a specific layout to form an infrared array. The infrared array overlaps with the projection plane in a direction perpendicular to the projection plane, allowing the user to perform gestures or other forms of interactive operations above the projection plane, and the projection device can recognize the user's actions by detecting changes in the infrared array.
[0042] It should be understood that if the infrared array is directly irradiated onto the projection plane, optical interference may occur. Therefore, such interference can be reduced by setting a small gap. Fig. 7A As shown, the multiple infrared emitters 7101 of the projection device 710 form an infrared array area AF, which is arranged above the projection plane of the projection device 710 (i.e., the plane where the projection area CD is located), and there is a small gap between the two areas. Fig. 7A and Figure 7B As shown, the infrared array area AF is also designed to be larger than the projection area CD projected by the micro-projector 7103 of the projection device 710 on its projection plane, so as to ensure that the infrared signal can fully cover the required interaction area.
[0043] In an embodiment of the present application, an inertial sensor on the stylus is used to determine the operating state of the stylus, and the working state of the corresponding infrared emitter is controlled in each touch state, so as to realize free switching of control modes in any scenario, reduce the dependence on physical buttons and virtual buttons, and improve the convenience of user interaction.
[0044] It should be noted that each implementation method of the present application can be freely combined, the order can be changed, or it can be executed separately, and does not need to rely on or depend on a fixed execution order.
[0045] In some embodiments, it also includes: The stylus pen acquires pressure sensing data of a pressure sensor, and the pressure sensor is arranged at the tip of the stylus pen; In response to the pressure sensing data exceeding a threshold, the first infrared emitter of the stylus pen is in an activatable state.
[0046] Specifically, a pressure sensor is built into the tip of the stylus. Figure 5 As shown, the pressure sensor can detect the pressure sensing data when the user presses the pen tip. A threshold is also set in the internal control program of the stylus. When the pressure sensing data exceeds the set threshold, the stylus will respond, that is, the response operation state is the pen tip operation state, and the first infrared emitter of the stylus enters an excitable state.
[0047] In some embodiments, the stylus determines the operation state of the stylus according to the inertial sensor data, including: The stylus determines the posture data of the stylus according to the inertial sensor data; The stylus determines an operation state of the stylus according to the gesture data.
[0048] Here, the stylus obtains the stylus posture data through the inertial sensor data measured by the internal inertial measurement unit IMU, such as Figure 5 The pitch angle (Pitch), roll angle (Roll) and yaw angle (Yaw) shown in the figure.
[0049] In one example, whether the stylus is in a pen tip operation state or a pen cap operation state may be determined according to a pitch angle (Pitch) of the stylus.
[0050] For example, two pitch angle ranges [Pp, Pp+△p] and [Ph, Ph+△h] are set, where Pp = 15° or 105°, △p = 60°; Ph = -15° or -105°, △h = -60°. When the pitch angle (Pitch) of the stylus is within [Pp, Pp+△p], the stylus is determined to be in the pen tip operation state; when the pitch angle (Pitch) of the stylus is within [Ph, Ph+△h], the stylus is determined to be in the pen cap operation state.
[0051] In some embodiments, it also includes: The projection device captures a first light spot formed by the reflection of infrared light from the first infrared emitter of the stylus; The projection device identifies the pen tip operation of the stylus pen according to the position information of the first light spot in the projection plane.
[0052] like Fig. 6A and Figure 6B As shown, the projection device 610 includes a plurality of infrared emitters 6101, an infrared camera 6102 and a micro projector 6103. The micro projector 6103 projects a projection area CD on its projection plane, the plurality of infrared emitters 6101 are in a closed state, i.e., do not emit infrared light, and the infrared camera captures infrared light under the infrared shooting area BE.
[0053] Continue to refer Fig. 6A and Figure 6B As shown, when the tip of the stylus pen 620 is pressed, the first infrared emitter of the stylus pen 620 will emit a beam of infrared light on the projection plane, and the infrared light will be reflected near the contact point X of the tip of the stylus pen 620, and the reflected infrared light will be captured by the infrared camera 6102. Since the infrared light reflected at the contact point X of the tip of the stylus pen is stronger than the infrared light in the background environment, the light signal intensity at the contact point X of the tip of the stylus pen will be significantly higher than the surrounding area without reflected light. In the output image of the infrared camera 6102, the area with high light signal intensity will be displayed as a light spot (such as a white light spot), that is, the first light spot.
[0054] The projection device can determine the specific position of the stylus tip on the projection area of the projection plane by analyzing the position information of the first light spot captured by the infrared camera. By comparing the position information of the first light spot on the projection plane with the content at the position information on the projection area of the projection plane, the specific operation of the stylus tip, such as drawing, clicking a virtual button, etc., can be identified.
[0055] In an embodiment of the present application, a first infrared emitter at the tip of the stylus pen emits infrared light, and an infrared camera captures a first light spot formed by reflected light of the infrared light, and accurately identifies the user's pen tip operation based on position information of the first light spot.
[0056] In some embodiments, it also includes: The projection device captures a second light spot formed by the reflection of infrared light in the infrared array; The projection device identifies the pen cap operation of the stylus pen according to the position information of the second light spot in the projection plane.
[0057] like Fig. 7A and Figure 7BAs shown, the projection device 710 includes a plurality of infrared emitters 7101, an infrared camera 7102 and a micro projector 7103. The micro projector 7103 projects a projection area CD on its projection plane, the plurality of infrared emitters 7101 emit infrared light to form an infrared array area AF, and the infrared camera captures infrared light under the infrared shooting area BE.
[0058] In the absence of obstructions, the infrared light in the infrared array is not reflected back to the infrared camera, so the infrared photosensitive element inside the infrared camera will not receive any light signal or weak light signal, and the infrared camera outputs an image of a completely black screen.
[0059] Continue to refer Fig. 7A and Figure 7B As shown, when there is an obstruction, that is, the cap of the stylus pen 720 clicks on the projection plane to block the local area L of the infrared array, the infrared light at the local area L where the cap of the stylus pen 720 clicks is reflected, and the reflected infrared light is captured by the infrared camera 7102. Since the infrared light reflected by the obstruction is stronger than the infrared light in the background environment, the light signal intensity at these points will be significantly higher than the surrounding areas without reflected light. In the output image of the infrared camera 7102, the area with high light signal intensity will be displayed as a light spot (such as a white light spot), which is the second light spot.
[0060] The controller inside the projection device will analyze the position information of the second light spot captured by the infrared camera in the projection area of the projection plane, compare the position information of the second light spot in the projection plane with the content at the position information on the projection area of the projection plane, and identify the specific operation of the stylus pen cap, such as erasing operation.
[0061] In an embodiment of the present application, by utilizing multiple second infrared emitters of the projection device to emit an infrared array, the infrared camera captures a second light spot formed by the reflected light of the infrared light in the infrared array, and accurately identifies the user's pen cap operation based on the position information of the second light spot.
[0062] Figure 8 The second flow chart of the control method provided in the embodiment of the present application is as follows: Figure 8 As shown, a control method is provided, including the following steps: step 810, step 820, step 830, step 840. The steps of the method flow are only a possible implementation of the present application.
[0063] Step 810: The projection device receives the first control signal sent by the stylus pen, wherein the first control signal is configured to indicate that the stylus pen is in a pen tip operation state; The first control signal is sent when the stylus pen is switched to the pen tip operation state. Here, the triggering method of the first control signal is the same as that in the above text, and will not be described in detail here.
[0064] The projection device receives a first control signal sent by the stylus pen, and determines that the stylus pen is in a pen tip operation state by decoding the first control signal.
[0065] Step 820: In response to the first control signal, the plurality of second infrared emitters of the projection device are in a closed state; It should be understood that the projection device includes a plurality of second infrared emitters, and these second infrared emitters are usually arranged in a certain array to form an infrared array on the projection plane of the projection device.
[0066] When the projection device receives the first control signal, the multiple second infrared emitters inside the projection device are controlled to be turned off to prevent the infrared arrays emitted by the multiple second infrared emitters from interfering with the infrared light emitted by the first infrared emitter of the stylus when the pen tip is in operation.
[0067] Step 830: The projection device receives the second control signal sent by the stylus pen, wherein the second control signal is configured to indicate that the stylus pen is in a pen cap operation state; The second control signal is sent when the stylus pen is switched to the pen cap operation state. Here, the triggering method of the second control signal is the same as that in the above text, and will not be described in detail here.
[0068] The projection device receives a second control signal sent by the stylus pen, and determines that the stylus pen is in a pen cap operation state by decoding the second control signal.
[0069] Step 840: In response to the second control signal, the plurality of second infrared emitters of the projection device are in an emitting state to form an infrared array, wherein, in a direction perpendicular to the projection plane formed by the projection device, the infrared array overlaps the projection plane.
[0070] When the projection device receives the second control signal, the plurality of second infrared emitters inside the projection device are controlled to turn on and emit infrared light to form an infrared array.
[0071] Here, the projection device can project a projection area on its projection plane, and the projection area is a content interface displayed on the projection plane. The projection plane can be a variety of flat surfaces, such as a desktop, a wall, a curtain, etc.
[0072] In order to realize the interactive function, the projection device has multiple second infrared emitters built in, which are arranged in a specific layout to form an infrared array. The infrared array overlaps the projection plane in a direction perpendicular to the projection plane, allowing the user to perform gestures or other forms of interactive operations above the projection plane, and the projection device can recognize the user's actions by detecting changes in the infrared array.
[0073] In an embodiment of the present application, a stylus pen is used to control the working states of multiple second infrared emitters of a projection device under control signals sent in each touch state, thereby achieving free switching of control modes in any scenario, reducing dependence on physical buttons and virtual buttons, and improving the convenience of user interaction.
[0074] In some embodiments, it also includes: The projection device captures a first light spot formed by the reflection of infrared light from the first infrared emitter of the stylus pen in the projection plane; The projection device identifies the pen tip operation of the stylus pen according to the position information of the first light spot in the projection plane.
[0075] When the tip of the stylus is pressed, the first infrared emitter of the stylus emits a beam of infrared light on the projection plane, and the infrared light is projected at the contact point of the stylus tip (such as Fig. 6A The infrared light is reflected near the point X shown in the figure, and the reflected infrared light is captured by the infrared camera. Since the infrared light reflected at the contact point of the pen tip is stronger than the infrared light in the background environment, the light signal intensity at the contact point of the pen tip will be significantly higher than that in the surrounding area without reflected light. In the output image of the infrared camera, the area with high light signal intensity will be displayed as a light spot (such as a white light spot), that is, the first light spot.
[0076] The projection device can determine the specific position of the stylus tip on the projection area of the projection plane by analyzing the position information of the first light spot captured by the infrared camera. By comparing the position information of the first light spot on the projection plane with the content at the position information on the projection area of the projection plane, the specific operation of the stylus tip, such as drawing, clicking a virtual button, etc., can be identified.
[0077] In an embodiment of the present application, a first infrared emitter at the tip of the stylus pen emits infrared light, and an infrared camera captures a first light spot formed by reflected light of the infrared light, and accurately identifies the user's pen tip operation based on position information of the first light spot.
[0078] In some embodiments, it also includes: The projection device captures a second light spot formed by the reflection of infrared light in the infrared array; The projection device identifies the pen cap operation of the stylus pen according to the position information of the second light spot in the projection plane.
[0079] In the absence of obstructions, the infrared light in the infrared array is not reflected back to the infrared camera, so the infrared photosensitive element inside the infrared camera will not receive any light signal or weak light signal, and the infrared camera outputs an image of a completely black screen.
[0080] When there is occlusion, that is, the stylus cap clicks on the projection plane and partially blocks the infrared array, the infrared light at the point where the stylus cap clicks is reflected, and the reflected infrared light is captured by the infrared camera. Since the infrared light reflected by the obstruction is stronger than the infrared light in the background environment, the light signal intensity at these points will be significantly higher than the surrounding areas without reflected light. In the output image of the infrared camera, the area with high light signal intensity will be displayed as a light spot (such as a white light spot), which is the second light spot.
[0081] The controller inside the projection device will analyze the position information of the second light spot captured by the infrared camera in the projection area of the projection plane, compare the position information of the second light spot in the projection plane with the content at the position information on the projection area of the projection plane, and identify the specific operation of the stylus pen cap, such as erasing operation.
[0082] In an embodiment of the present application, by utilizing multiple second infrared emitters of the projection device to emit an infrared array, the infrared camera captures a second light spot formed by the reflected light of the infrared light in the infrared array, and accurately identifies the user's pen cap operation based on the position information of the second light spot.
[0083] The embodiment of the present application further provides a control system, including a stylus pen and a projection device, wherein the stylus pen is configured as follows: determining an operating state of the stylus according to inertial sensor data; In response to the operating state being a pen tip operating state, the first infrared emitter of the stylus pen is in an excitable state and generates a first control signal corresponding to the pen tip operating state; In response to the operating state being a pen cap operating state, the first infrared emitter of the stylus pen is in an unexcitable state and generates a second control signal corresponding to the pen cap operating state; The projection device is configured as follows: The projection device receives the first control signal sent by the stylus; In response to the first control signal, the plurality of second infrared emitters of the projection device are in a closed state; The projection device receives the second control signal sent by the stylus; In response to the second control signal, the plurality of second infrared emitters of the projection device are in an emission state to form an infrared array, wherein in a direction perpendicular to a projection plane formed by the projection device, the infrared array overlaps the projection plane.
[0084] In this embodiment, the stylus pen and the projection device in the control system are the same as the stylus pen and the projection device in the above embodiment, and will not be described in detail here.
[0085] An embodiment of the present application also provides a projection device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the methods provided by the above-mentioned method embodiments when executing the program.
[0086] On the other hand, the present application also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the methods provided by the above-mentioned method embodiments.
[0087] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the methods provided by the above-mentioned method embodiments.
[0088] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0089] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A control method, comprising: The stylus determines the operation state of the stylus according to the inertial sensor data; In response to the operating state being a pen tip operating state, the first infrared emitter of the stylus pen is in an excitable state and generates a first control signal corresponding to the pen tip operating state; The projection device receives the first control signal sent by the stylus pen; In response to the first control signal, the plurality of second infrared emitters of the projection device are in a closed state; In response to the operating state being a pen cap operating state, the first infrared emitter of the stylus pen is in an unexcitable state and generates a second control signal corresponding to the pen cap operating state; The projection device receives the second control signal sent by the stylus; In response to the second control signal, the plurality of second infrared emitters of the projection device are in an emission state to form an infrared array, wherein in a direction perpendicular to a projection plane formed by the projection device, the infrared array overlaps the projection plane.
2. The control method according to claim 1, further comprising: The stylus pen acquires pressure sensing data of a pressure sensor, and the pressure sensor is arranged at the tip of the stylus pen; In response to the pressure sensing data exceeding a threshold, the first infrared emitter of the stylus pen is in an activatable state.
3. The control method according to claim 1, wherein: The stylus determines an operation state of the stylus according to inertial sensor data, including: The stylus determines the posture data of the stylus according to the inertial sensor data; The stylus determines an operation state of the stylus according to the gesture data.
4. The control method according to claim 1, further comprising: The projection device captures a first light spot formed by the reflection of infrared light from the first infrared emitter of the stylus; The projection device identifies the pen tip operation of the stylus pen according to the position information of the first light spot in the projection plane.
5. The control method according to claim 1, further comprising: The projection device captures a second light spot formed by the reflection of infrared light in the infrared array; The projection device identifies the pen cap operation of the stylus pen according to the position information of the second light spot in the projection plane.
6. A control method, comprising: The projection device receives the first control signal sent by the stylus pen, wherein the first control signal is configured to indicate that the stylus pen is in a pen tip operation state; In response to the first control signal, the plurality of second infrared emitters of the projection device are in a closed state; The projection device receives the second control signal sent by the stylus pen, wherein the second control signal is configured to indicate that the stylus pen is in a pen cap operation state; In response to the second control signal, the plurality of second infrared emitters of the projection device are in an emission state to form an infrared array, wherein in a direction perpendicular to a projection plane formed by the projection device, the infrared array overlaps the projection plane.
7. The control method according to claim 6, further comprising: The projection device captures a first light spot formed by the reflection of infrared light from the first infrared emitter of the stylus pen in the projection plane; The projection device identifies the pen tip operation of the stylus pen according to the position information of the first light spot in the projection plane.
8. The control method according to claim 6, further comprising: The projection device captures a second light spot formed by the reflection of infrared light in the infrared array; The projection device identifies the pen cap operation of the stylus pen according to the position information of the second light spot in the projection plane.
9. A control system, comprising a stylus pen and a projection device, wherein the stylus pen is configured to: determining an operating state of the stylus according to inertial sensor data; In response to the operating state being a pen tip operating state, the first infrared emitter of the stylus pen is in an excitable state and generates a first control signal corresponding to the pen tip operating state; In response to the operating state being a pen cap operating state, the first infrared emitter of the stylus pen is in an unexcitable state and generates a second control signal corresponding to the pen cap operating state; The projection device is configured as follows: The projection device receives the first control signal sent by the stylus; In response to the first control signal, the plurality of second infrared emitters of the projection device are in a closed state; The projection device receives the second control signal sent by the stylus; In response to the second control signal, the plurality of second infrared emitters of the projection device are in an emission state to form an infrared array, wherein in a direction perpendicular to a projection plane formed by the projection device, the infrared array overlaps the projection plane.
10. A projection device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 5 or any one of claims 6 to 8 is implemented.
11. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 5 or any one of claims 6 to 8 is implemented.