A multi-mode projection interaction system integrating dual cameras and TOF

Abstract

This utility model relates to the field of intelligent projection interaction technology, specifically to a multi-mode projection interaction system integrating dual cameras and Time-of-Flight (TOF), including a projection device and a laser pointer. The projection device comprises a housing, a projection optical engine, a camera, a TOF sensor, a main control chip, and a processing module, with each component connected according to its function. The laser pointer includes a pen barrel, a laser module, a Bluetooth module, and a microcontroller unit; the laser module emits laser light through a laser hole at the front end of the pen barrel. During projection, the camera acquires the projected image and simultaneously captures the position of the laser beam projected by the laser pointer, enabling the user to operate on the projected image using the laser pointer. The TOF sensor measures the distance from the projection optical engine to the projection area, and the processing module filters and identifies the laser beam based on this distance data. This system integrates dual cameras and TOF, improving the convenience and accuracy of interaction.

Description

Technical Field

[0001] This utility model relates to the field of intelligent projection interaction technology, and in particular to a multi-mode projection interaction system integrating dual cameras and TOF. Background Technology

[0002] Driven by the trend towards portable digital office work, smart projection devices, with their flexible mobility, have become core tools for cross-scenario information display. However, the lag in interactive experience still limits their effectiveness. The shortcomings of existing devices in remote control and real-time handwriting annotation are particularly evident: traditional laser pointers can mostly only achieve single-point pointing and cannot complete complex remote operations, such as page turning and content selection. Users still need to frequently approach the device or rely on additional controllers. In scenarios requiring real-time annotation, even if some laser pointers claim to have handwriting functions, insufficient positioning accuracy and significant handwriting delay often result in annotation effects that deviate significantly from actual writing habits, making it difficult to meet the precise needs of quickly highlighting key points in meetings and deriving formulas in real time during teaching.

[0003] This limitation puts users in a dilemma: if they prioritize the convenience of remote control, they must sacrifice the accuracy of handwritten annotations; if they emphasize the handwriting experience, they have to rely on external devices, which contradicts the core requirement of smart projectors: "portability and efficiency." For example, in cross-departmental collaborative meetings, when the presenter is using a regular laser pointer to indicate the projected content, if they need to modify a section of text or add a diagram, they must pause the presentation, walk to the device, or switch to an external writing tablet. This not only interrupts their train of thought but also reduces the continuity of the meeting. Therefore, how to build a multi-mode interactive system that combines precise remote control with a natural handwriting experience based on the collaboration between the projector and the laser pointer has become a key direction for overcoming existing technological bottlenecks and adapting to the needs of modern office scenarios. Utility Model Content

[0004] To address the technical challenge of constructing a multi-mode interactive system that combines precise remote control with a natural handwriting experience based on the collaboration of a projection device and a laser pointer, this invention provides a multi-mode projection interactive system integrating dual cameras and a Time-of-Flight (TOF) sensor.

[0005] Therefore, the present invention provides the following technical solution: A multi-mode projection interaction system integrating dual cameras and Time-of-Flight (TOF) includes a projection device and a laser pen. The projection device includes a housing, a projection optical engine, a camera, a TOF sensor, a main control chip, and a processing module. The projection optical engine, camera, and TOF sensor are mounted on the housing, while the main control chip and processing module are mounted inside the housing. The projection optical engine, camera, and TOF sensor are all connected to the main control chip, which is also connected to the processing module. The laser pen includes a pen barrel, a laser module, and a microcontroller unit. The pen barrel has a hollow structure, with the laser module and microcontroller unit installed inside. The laser module is connected to the microcontroller unit, and a pen tip sleeve is installed at the front end of the pen barrel. The pen tip sleeve has a laser hole through which the laser emitted by the laser module can pass.

[0006] Furthermore, the laser pointer also includes a Bluetooth module and capacitive touch-sensitive buttons and / or mechanical buttons for input control. The Bluetooth module, capacitive touch-sensitive buttons, and mechanical buttons are all connected to the microcontroller unit. The Bluetooth module can interact with the main control chip, and the capacitive touch-sensitive buttons and mechanical buttons are all mounted on the pen barrel.

[0007] Furthermore, the pen tip sleeve is detachably mounted on the front end of the pen barrel, and a pressure sensor is installed between the pen tip sleeve and the pen barrel.

[0008] Furthermore, the laser emitted by the laser module is an invisible infrared laser.

[0009] Furthermore, an IRCUT dual filter switcher is installed on the optical path between the camera lens and the image sensor. The filters in the IRCUT dual filter switcher include a visible light filter and an infrared light filter.

[0010] Furthermore, a protective cover is slidably installed on the pen barrel along its length, and the capacitive touch button 6 and the mechanical button are located on the path of the protective cover's movement.

[0011] Furthermore, the mechanical buttons include a power switch, a page-turning button, and / or a mouse style switching button.

[0012] Furthermore, a rechargeable battery is installed inside the pen barrel, and a USB-C interface for charging the rechargeable battery is installed on the pen barrel.

[0013] Advantages and positive effects of this utility model.

[0014] This invention relates to a projection device that uses a camera to capture the projected image and the laser emitted by a laser module, enabling precise remote control. By pressing the laser pointer tip onto the projected image area and using a pressure sensor to control the opening and closing of the laser module, a natural handwriting experience is achieved. This solves the technical problem in existing technologies of how to construct a multi-mode interactive system that combines precise remote control with a natural handwriting experience based on the collaboration of a projection device and a laser pointer. The projection device projects an image onto the projection area using a projection engine, captures the projected image using a camera, and projects a laser beam onto the projected image area using the laser pointer's laser module. A TOF sensor measures the distance from the projection engine to the projection area, filters the laser beam, and the camera captures the laser position. Users can then directly operate on the projected image using the laser pointer, improving the convenience and accuracy of user interaction with the projection system. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this utility model or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this utility model. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 The present invention provides a structural diagram of a projection device for a multi-mode projection interaction system integrating dual cameras and TOF.

[0017] Figure 2 The schematic diagram of a projection device for a multi-mode projection interaction system integrating dual cameras and TOF provided by this utility model.

[0018] Figure 3 This utility model provides a first structural diagram of a laser pen that integrates dual cameras and a TOF multi-mode projection interaction system.

[0019] Figure 4 This utility model provides a second structural diagram of a laser pen that integrates dual cameras and a TOF multi-mode projection interaction system.

[0020] Figure 5 The present invention provides a third structural diagram of a laser pen that integrates dual cameras and a TOF multi-mode projection interaction system.

[0021] Figure 6 The present invention provides a fourth structural diagram of a laser pen that integrates dual cameras and a TOF multi-mode projection interaction system.

[0022] Figure 7This utility model provides a schematic diagram of a laser pointer that integrates dual cameras and a TOF multi-mode projection interaction system.

[0023] In the picture: 1. Housing; 2. Projector engine; 3. Camera; 4. Pen barrel; 5. Laser hole; 6. Capacitive touch button; 7. Mechanical button; 8. Protective cover; 9. USB-C interface; 10. Pen tip cover. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0025] This utility model provides a multi-mode projection interaction system integrating dual cameras and TOF, including a projection device and a laser pointer, such as... Figures 1-2 As shown, the projection device includes a housing 1, a projection optical engine 2, a camera 3, a TOF sensor, a main control chip, and a processing module. The projection optical engine 2, the camera 3, and the TOF sensor are mounted on the housing 1, and the main control chip and the processing module are mounted inside the housing 1. The projection optical engine 2, the camera 3, and the TOF sensor are all connected to the main control chip, and the main control chip is connected to the processing module.

[0026] The TOF sensor is used to measure the distance from the projector's optical engine to the projection area. Lasers have high focusing properties, so the physical size of the laser spot produced by a user using a laser pointer at different distances varies only slightly. Cameras, on the other hand, exhibit a near-large-area-and-far-small-area characteristic; when the device is close to the projection area (wall or screen), the laser spot occupies a large area in the captured image, and vice versa. During the projection device calibration phase, the TOF sensor obtains the distance *d* between the projection device and the projection area. The one-to-one correspondence between distance *d* and the laser spot pixel diameter *r* is calibrated at different distance points, constructing a *dr* parameter library. In the usage scenario, after the projection device is placed, the projector projects the image onto the projection area. The TOF sensor obtains the actual distance between the projection device and the projection area, and the appropriate laser spot pixel diameter parameter is selected based on the *dr* parameter library to filter the laser spot and improve operational accuracy.

[0027] An IRCUT dual filter switcher is installed in the optical path between the camera lens and the image sensor. The filters in the IRCUT dual filter switcher include a visible light filter and an infrared light filter. The lens and image sensor are inherent components of the camera, both of which are existing standardized components and can be directly adapted for use. The installed IRCUT dual filter switcher is also an existing device. An IRCUT dual filter switcher typically consists of two filters, a power mechanism, a housing, and a circuit control board. The circuit control board controls the movement of the power mechanism to achieve the switching of the two filters.

[0028] like Figures 3-7 As shown, the laser pointer includes a pen barrel 4, a laser module, a Bluetooth module, and a microcontroller unit. The pen barrel 4 has a hollow structure, and the laser module, Bluetooth module, and microcontroller unit are installed inside the pen barrel 4. Both the laser module and the Bluetooth module are connected to the microcontroller unit. The Bluetooth module can interact with the main control chip. A pen tip sleeve 10 is installed at the front end of the pen barrel 4 in a threaded and detachable manner. The pen tip sleeve 10 has a laser hole 5, and the laser emitted by the laser module can pass through the laser hole 5.

[0029] The laser pointer also includes a capacitive touch button 6, a mechanical button 7, and a pressure sensor for input control. The capacitive touch button 6, the mechanical button 7, and the pressure sensor are all connected to the microcontroller unit. The capacitive touch button 6 and the mechanical button 7 are both mounted on the pen barrel 4, and the pressure sensor is mounted between the pen barrel 4 and the pen tip sleeve 10.

[0030] The laser emitted by the laser module is an invisible infrared laser with a wavelength of 940nm-980nm, and the physical diameter of the circular spot formed after projection is 3-5mm. The microcontroller unit controls the operation of the laser module and Bluetooth module, receiving input signals from the capacitive touch button 6, mechanical button 7, and pressure sensor. Mechanical button 7 includes a power button, a page-turning button, and a mouse style switching button. The page-turning button controls the page turning of the projected image. When the projection device acquires the position of the laser spot within the projection area, it can display the laser spot as a mouse cursor on the projected image. The mouse style switching button allows users to change the mouse cursor style for better interactive effects.

[0031] A protective cover 8 is slidably mounted on the pen body 4 along its length. The capacitive touch button 6 and the mechanical button 7 are located along the path of the protective cover 8. A rechargeable battery is installed inside the pen body 4, and a USB-C port 9 for charging the rechargeable battery is mounted on the pen body 4.

[0032] Working principle: Before the interactive system is put into use, a TOF sensor on the projection device housing measures the distance d from the projection optical engine to the projection area. Simultaneously, invisible infrared laser light is projected at different distance points using a laser pointer. Due to the strong focusing ability of the laser, the physical spot size difference is small at different distances. However, the camera exhibits a "near-larger, far-smaller" characteristic; the closer the distance, the larger the captured spot pixel diameter r, and vice versa. The staff maps d and r to each distance point, constructing a dr parameter library and storing it in the device's processing module.

[0033] In actual use, the projection device is first placed. The main control chip controls the projection optical engine to project the image onto the projection area. The TOF sensor simultaneously obtains the actual distance between the current device and the projection area. The processing module calls the DR parameter library to match the appropriate laser spot pixel diameter, thereby filtering the image captured by the camera, eliminating stray light interference, and improving the spot positioning accuracy. The IRCUT dual filter switcher first switches to the visible light filter. At this time, the camera focuses on the projected image, and the projection optical engine performs keystone correction. After focusing and keystone correction are completed, the IRCUT dual filter switcher switches to the infrared light filter. At this time, the camera can capture the laser spot image within the projected image.

[0034] Users project laser light onto the projected image area using the laser pointer's laser module. After the camera captures the laser spot, the main control chip and processing module work together to map the coordinates of the laser spot from the lens coordinate system to the system UI coordinate system using a perspective transformation matrix. This allows users to interact with the projected image using the laser pointer. The perspective transformation matrix is ​​constructed based on the coordinates of the four corner markers in the lens coordinate system and the UI coordinates of the projection interaction device.

[0035] Slide the protective cover 8 to the end of the pen barrel, touch the capacitive touch button 6, and the laser pen enters air mouse mode. At this time, the laser module emits a laser beam to the projection area, enabling long-distance operation. Moving the laser pen will move the mouse cursor displayed on the projected screen. When the laser spot is in a writing area, pressing and releasing the capacitive touch button 6 will draw dots in that area. Continuing to press and move the button will enter writing mode, allowing you to draw lines within the writing area. When there is a button or icon in the laser spot area, pressing and releasing the capacitive touch button 6 will generate a click event, clicking the button or icon. Continuing to press and move the laser pen will drag the button or icon. If the projection device cannot detect the laser spot within a set time on the projected screen, the air mouse mode will automatically turn off, or the air mouse mode will automatically turn off when you stop touching the capacitive touch button 6.

[0036] Slide the protective cover 8 to the head of the pen and press the laser pointer tip 10 onto the projected image area. The pressure sensor is pressed by the tip 10 and the pen body 4, triggering the laser module to activate and enter handwriting mode. The projection device acquires the position of the laser spot, allowing the user to operate within the projection area at close range. When the laser spot area is a writing area, pressing and lifting the laser pointer will draw dots in that area. Continuing to press and move the pointer will enter writing mode, allowing the user to draw lines within the writing area. When the laser spot area contains a button or icon, pressing and lifting the laser pointer will trigger a click event, allowing the user to click the button or icon. Continuing to press and move the laser pointer will drag the button or icon. When the laser pointer moves away from the projected image and the pressure sensor between the tip and the pen body is no longer pressed, the handwriting mode automatically deactivates.

[0037] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this utility model, and are not intended to limit it. Although the utility model has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this utility model.

Claims

1. A multi-mode projection interaction system integrating dual cameras and TOF, characterized in that, The device includes a projection device and a laser pen. The projection device includes a housing (1), a projection optical engine (2), a camera (3), a TOF sensor, a main control chip, and a processing module. The projection optical engine (2), the camera (3), and the TOF sensor are installed on the housing (1). The main control chip and the processing module are installed inside the housing (1). The projection optical engine (2), the camera (3), and the TOF sensor are all connected to the main control chip. The main control chip is connected to the processing module. The laser pen includes a pen barrel (4), a laser module, and a microcontroller unit. The pen barrel (4) has a hollow structure. The laser module and the microcontroller unit are installed inside the pen barrel (4). The laser module is connected to the microcontroller unit. A pen tip sleeve (10) is installed at the front end of the pen barrel (4). A laser hole (5) is provided on the pen tip sleeve (10). The laser emitted by the laser module can pass through the laser hole (5).

2. The multi-mode projection interaction system integrating dual cameras and TOF as described in claim 1, characterized in that, The laser pointer also includes a Bluetooth module and a capacitive touch button (6) and / or a mechanical button (7) for input control. The Bluetooth module, the capacitive touch button (6) and the mechanical button (7) are all connected to the microcontroller unit. The Bluetooth module can interact with the main control chip. The capacitive touch button (6) and the mechanical button (7) are both installed on the pen barrel (4).

3. The multi-mode projection interaction system integrating dual cameras and TOF as described in claim 1, characterized in that, The pen tip sleeve (10) is detachably installed at the front end of the pen barrel (4), and a pressure sensor is installed between the pen tip sleeve (10) and the pen barrel (4).

4. The multi-mode projection interaction system integrating dual cameras and TOF as described in claim 1, characterized in that, The laser emitted by the laser module is an invisible infrared laser.

5. A multi-mode projection interaction system integrating dual cameras and TOF according to claim 1, characterized in that, An IRCUT dual filter switcher is installed on the optical path between the lens and the image sensor of the camera (3). The filters in the IRCUT dual filter switcher include a visible light filter and an infrared light filter.

6. A multi-mode projection interaction system integrating dual cameras and TOF according to claim 2, characterized in that, A protective cover (8) is slidably installed on the pen (4) along the length of the pen (4), and the capacitive touch button (6) and the mechanical button (7) are located on the path of the protective cover (8).

7. A multi-mode projection interaction system integrating dual cameras and TOF according to claim 2, characterized in that, The mechanical button (7) includes a power button, a page-turning button, and / or a mouse style switching button.

8. A multi-mode projection interaction system integrating dual cameras and TOF according to claim 1, characterized in that, A rechargeable battery is installed inside the pen (4), and a USB-C interface (9) for charging the rechargeable battery is installed on the pen (4).

Images