Remote controller control method for projector, projection system and storage medium

By obtaining the main transformation matrix between the shooting coordinate system of the projector remote control and the coordinate system of the projected image, the position of the light spot is determined and the brightness is reduced or the laser is turned off, thus solving the problem that the laser pointer may shine into people's eyes and realizing safe interactive control of the projector.

CN119011793APending Publication Date: 2024-11-22深圳市当智科技有限公司
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410968455.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

When using a laser pointer to interact with projected images, if someone stands next to the projected image, the laser may inadvertently shine into their eyes, causing injury. Current technology is not effective in preventing such situations.

Method used

By obtaining the main transformation matrix between the shooting coordinate system and the original projection coordinate system, the position of the light spot in the projection mapping area is determined, and the brightness of the light spot is reduced or the laser emission is turned off when the light spot may shine into the human eye.

Benefits of technology

It effectively prevents laser damage to the human eye, ensuring that the light spot only works within the projected image area, thus protecting user safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119011793A_ABST
    Figure CN119011793A_ABST
Patent Text Reader

Abstract

The invention discloses a remote controller control method for a projector, a projection system and a storage medium. The remote controller control method comprises the following steps: acquiring a main conversion matrix between a shooting coordinate system and a projection original image coordinate system; based on the main transformation matrix and a projection area in the projection original image coordinate system, determining a projection mapping area corresponding to the projection area in the shooting coordinate system; calculating a theoretical light spot coordinate in the shooting coordinate system, and when the theoretical light spot coordinate is located outside the projection mapping area; or when the theoretical light spot coordinate is located in the projection mapping area, but the light spot cannot be recognized through the camera, the remote controller is indicated to reduce the brightness of the emitted light spot. The device can prevent laser from hurting human eyes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of projection equipment, and in particular to a remote control method for a projector, a projection system and a storage medium. Background Art

[0002] When it comes to how to emphasize specific content in the projection screen, the user generally uses a remote controller with a laser emission module to illuminate the specific content in the projection screen, or can control the content where the laser spot in the projection screen is located by pressing buttons on the remote controller.

[0003] However, when using a laser pen to achieve interaction with the projection screen, if someone is standing next to the projection screen, the user operating the laser pen may inadvertently move the direction of the laser pen out of the projection screen and shine it into the person's eyes; however, high-power lasers cannot be directed directly into the human eye, and even low-power lasers can cause cumulative damage if they continue to shine into the human eye. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a remote control method for a projector, a projection system and a storage medium, which can prevent laser from causing damage to human eyes.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention discloses a remote control method for a projector, comprising the following steps:

[0007] Get the main transformation matrix between the shooting coordinate system and the projection original image coordinate system;

[0008] Based on the main transformation matrix and the projection area in the projection original image coordinate system, determining a projection mapping area corresponding to the projection area in the shooting coordinate system;

[0009] The theoretical light spot coordinates in the shooting coordinate system are calculated, and when the theoretical light spot coordinates are outside the projection mapping area; or when the theoretical light spot coordinates are within the projection mapping area but the light spot cannot be recognized by the camera, the remote control is instructed to reduce the brightness of the emitted light spot.

[0010] Preferably, before calculating the theoretical light spot coordinates in the shooting coordinate system, the following steps are also included:

[0011] Acquire the projection distance and the pitch angle of the projector, and calculate the initial pitch angle range of the remote control based on the projection distance, the pitch angle of the projector, and a preset remote control usage distance;

[0012] Acquire the current pitch angle of the remote controller, and determine whether the current pitch angle of the remote controller is within the initial pitch angle range;

[0013] When the current pitch angle of the remote controller is within the initial pitch angle range, the remote controller is driven to emit a light spot at a transmission power lower than a first threshold.

[0014] Preferably, when the current pitch angle of the remote controller is within the initial pitch angle range, after driving the remote controller to emit a light spot with a transmission power lower than a first threshold, the method further includes the following steps:

[0015] It is determined whether the light spot emitted by the remote controller is recognized by the camera, and if so, the remote controller is instructed to increase the brightness of the emitted light spot.

[0016] Preferably, the calculating of the theoretical light spot coordinates in the shooting coordinate system comprises:

[0017] Acquire a shooting picture, determine the starting light spot coordinates from the shooting picture, and acquire the first real-time motion parameter of the remote controller; determine the theoretical light spot coordinates in the shooting coordinate system according to the starting light spot coordinates and the first real-time motion parameter of the remote controller, wherein the shooting coordinate system is established based on the shooting picture; or,

[0018] Acquire a captured image and determine the starting light spot coordinates from the captured image; establish a first spatial coordinate system including the captured image based on the current projection distance of the projector, the first spatial attitude angle of the projector and the preset viewing angle of the captured image; obtain the starting coordinates and starting orientation of the remote control in the first spatial coordinate system based on the starting light spot coordinates, the distance of the remote control from the wall and the second spatial attitude angle of the remote control; acquire the second real-time motion parameters of the remote control, and determine the theoretical light spot coordinates in the captured image coordinate system based on the starting coordinates and starting orientation of the remote control in the first spatial coordinate system and the second real-time motion parameters of the remote control. The captured image coordinate system is established based on the captured image, and the distance of the remote control from the wall refers to the distance between the remote control and the target projection surface.

[0019] Preferably, the calculating of the theoretical light spot coordinates in the shooting coordinate system further comprises: when the spatial position change of the remote controller exceeds a second threshold, re-determining the starting light spot coordinates from the shooting picture.

[0020] Preferably, after instructing the remote controller to reduce the brightness of the emitted light spot when the theoretical light spot coordinates are outside the projection mapping area, the method further includes: instructing the remote controller to increase the brightness of the emitted light spot when the theoretical light spot coordinates are within the projection mapping area.

[0021] Preferably, the method of acquiring the main transformation matrix between the shooting coordinate system and the projection original image coordinate system includes: acquiring a first sub-transformation matrix preset between the infrared shooting screen and the visible light shooting screen; projecting a feature map onto the target projection surface, and acquiring a visible light shooting screen, wherein the visible light shooting screen contains the feature map; obtaining a second sub-transformation matrix based on the coordinates of the feature map in the projection original image and the visible light shooting screen respectively; and obtaining the main transformation matrix based on the first sub-transformation matrix and the second sub-transformation matrix.

[0022] Preferably, the shooting coordinate system is an infrared shooting screen coordinate system, and the conversion matrix is ​​composed of a first sub-conversion matrix and a second sub-conversion matrix. The first sub-conversion matrix is ​​a conversion matrix between the infrared shooting screen coordinate system and the visible light shooting screen coordinate system, and the second sub-conversion matrix is ​​a conversion matrix between the visible light shooting screen coordinate system and the projection original image coordinate system after trapezoidal correction is performed on the projection area.

[0023] In a second aspect, the present invention discloses a projection system, including a projector and a remote control, wherein the projector includes a processor and a memory, wherein the memory stores a computer program, and the computer program can be read and executed by the processor to implement the remote control method as described in the first aspect.

[0024] In a third aspect, the present invention discloses a storage medium storing a computer program, wherein the computer program is configured to be executable by a processor to execute the remote controller control method described in the first aspect.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention proposes a remote control method for a projector, which determines the projection mapping area in the shooting coordinate system based on the main transformation matrix between the shooting coordinate system and the projection original image coordinate system, and can further determine whether the theoretical light spot coordinates in the shooting coordinate system are in the projection mapping area, and when the theoretical light spot coordinates are outside the projection mapping area, or when the theoretical light spot coordinates are within the projection mapping area but cannot be recognized by the camera, it is considered that the light spot may irradiate the human eye, and the remote control is instructed to reduce the brightness of its light spot, thereby preventing the laser from causing damage to the human eye. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a schematic diagram of the hardware structure of the remote control;

[0027] Figure 2 It is a schematic diagram of the hardware structure of the projector;

[0028] Figure 3 is a flow chart of a remote control method for a projector disclosed in Embodiment 1 of the present invention;

[0029] Figure 4 It is a schematic diagram of the mapping relationship between the infrared shooting screen coordinate system, the visible light shooting screen coordinate system and the projection original image coordinate system;

[0030] Figure 5 It is a schematic diagram of the movement of light spots in the infrared shooting picture;

[0031] Figure 6 It is a schematic diagram of the position relationship of the remote control relative to the target projection surface;

[0032] Figure 7 It is a schematic diagram of the positional relationship between the projector, remote control and infrared shooting screen;

[0033] Figure 8 A schematic diagram of the hardware structure of a remote controller according to another embodiment;

[0034] Fig. 9 is the remote control's pitch field of view;

[0035] Fig.10 It is a schematic diagram of the positional relationship between the projector and the remote control relative to the projected image. DETAILED DESCRIPTION

[0036] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope and application of the present invention.

[0037] It should be noted that when a component is referred to as being "fixed to" or "disposed on" another component, it can be directly on the other component or indirectly on the other component. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, connection can be used for both fixing and circuit / signal communication.

[0038] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0039] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0040] Embodiment 1 of the present invention discloses a remote control method for a projector, wherein the hardware structure of the remote control 10 is shown in FIG. Figure 1 As shown, the hardware structure diagram of the projector 20 is as follows Figure 2 shown.

[0041] The remote controller 10 mainly includes an infrared laser 11, a power drive circuit 12, a Bluetooth remote control SOC (System on Chip) chip 13 and a first IMU (Inertial Measurement Unit) unit 14; wherein the power drive circuit 12 is used to drive the infrared laser 11, and the infrared laser 11 emits infrared light. In an interactive method, after identifying the corresponding position of the infrared light spot in the projection screen, the projection screen displays a bright spot at the position to represent the position of the infrared light spot, thereby visualizing the infrared light spot. The Bluetooth remote control SOC chip 13 can control the on and off of the infrared laser 11 through the power drive circuit 12, as well as the on time length, that is, the pulse width of the on PWM waveform; the first IMU unit 14 is used to detect the posture of the remote controller 10; the remote controller 10 can also establish a Bluetooth connection with the projector 20 through the Bluetooth remote control SOC chip 13. Specifically, the Bluetooth remote control SOC chip 13 is an integrated circuit with a dedicated purpose, which integrates the processor (CPU), graphics processing unit (GPU), memory controller (Memory Controller), wireless radio frequency chip (RF) and other modules; the remote control 10 generally extracts and calculates various data through the CPU, and sends and receives wireless signals through the wireless radio frequency chip (including the Bluetooth module). Among them, the remote control 10 is equipped with a first IMU unit 14, and the inertial sensor in the first IMU unit 14 can detect the angular velocity and linear acceleration of the remote control itself, and then the angle change and displacement change of the remote control itself can be calculated by the time integration method.

[0042] The projector 20 mainly includes a first camera 21, a second camera 22, an optical machine 23, a SOC chip 24, a BT module 25, and a second IMU unit 26. Among them, the first camera 21 is a visible light camera that can shoot RGB three-color visible light images; the second camera 22 is an infrared camera with an infrared filter, which can image infrared light of a specific band; the BT (BlueTooth) module 25 is used to establish a Bluetooth connection with the remote control 10, and is used for the key value / voice data / status synchronous communication of the remote control; the second IMU unit 26 is used to detect changes in the pitch angle and rotation angle of the projector 20, and is used for calculating the automatic trapezoidal / autofocus function of the projector 20 itself. Among them, the filter band of the second camera 22 can be a narrow band, which only passes infrared rays of a specific band, such as 980nm and 850nm; it can also be a broadband filter, which filters out the visible light band and retains the near-infrared band, such as filtering out visible light less than 780nm and retaining the infrared band above 780nm; the light spot scattered by the infrared laser emitted by the infrared laser 11 on the wall / curtain can be easily imaged by the infrared camera (second camera 22), and the moving scattered light spot can be compared by the previous and next two frames of images, and the trajectory of the infrared light spot can be identified.

[0043] It should be noted that in some embodiments, the remote control may also emit a visible light spot via a visible light laser, and the projector may directly identify the position of the visible light spot via a visible light camera.

[0044] The remote control 10 is connected to the projector 20 via Bluetooth communication, and can report the key value of the remote control to the projector host through the Bluetooth module. After the key value is reported to the host, the projector executes the instruction corresponding to the key value, such as the "confirm" instruction. Among them, a certain key on the hardware of the remote control 10, such as the confirmation key (OK key), is defined as a touch control key. When the infrared spot of the remote control 10 points to a certain position of the projection screen, the second camera 22 of the projector 20 recognizes the spot position and displays a prompt cursor at the corresponding coordinate position of the projection screen; when the OK key is pressed, it indicates that a key press event is generated at the coordinate, when the OK key is pressed and moved, it indicates a press and move event, and when the OK key pops up, it indicates a key lift event.

[0045] like Figure 3 As shown, the remote control method for a projector disclosed in Embodiment 1 of the present invention includes the following steps:

[0046] S1: Obtain the main transformation matrix between the shooting coordinate system and the projection original image coordinate system;

[0047] Among them, the shooting coordinate system is a coordinate system established based on the shooting picture. For example, the shooting coordinate system can use a vertex of the shooting picture as the origin of the coordinate system, and the XOZ plane coincides with the shooting picture. The coordinates of other vertices of the shooting picture in the shooting coordinate system are set according to the resolution value of the shooting picture. The projection original image coordinate system is a coordinate system established based on the picture to be projected loaded in the projector. Similarly, the projection original image coordinate system can use a vertex of the projection original image as the origin of the coordinate system, and the XOZ plane coincides with the projection original image. The coordinates of other vertices of the projection original image in the projection original image coordinate system are set according to the resolution value of the projection original image.

[0048] In this embodiment, the light spot emitted by the remote control is an invisible light spot, such as an infrared light spot. At this time, the second camera 22 can capture the infrared light spot. Therefore, the shooting picture in this step corresponds to the infrared shooting picture, and the shooting coordinate system is the infrared shooting picture coordinate system established based on the infrared shooting picture. The main conversion matrix is ​​composed of a first sub-conversion matrix and a second sub-conversion matrix. The first sub-conversion matrix is ​​a conversion matrix between the infrared shooting picture coordinate system and the visible light shooting picture coordinate system, which represents the coordinate conversion relationship between the infrared shooting picture and the visible light shooting picture for the same light spot position; the second sub-conversion matrix is ​​a conversion matrix between the visible light shooting picture coordinate system and the projection original image coordinate system after trapezoidal correction is performed on the projection area, which represents the conversion relationship between a coordinate in the visible light shooting picture and its mapping coordinate in the projection original image. Furthermore, step S1 includes: obtaining a first sub-conversion matrix preset between the infrared shooting picture and the visible light shooting picture; projecting a feature map onto the target projection surface, and obtaining a visible light shooting picture, the visible light shooting picture containing the feature map; obtaining a second sub-conversion matrix based on the coordinates of the feature map in the original projection image and the visible light shooting picture respectively; obtaining a main conversion matrix based on the first sub-conversion matrix and the second sub-conversion matrix.

[0049] The infrared camera is used to capture the light spot scattered by the low-power infrared laser, while the visible light camera is used for the auto-focus and auto-trapezoidal functions of the projection. In order to obtain the correspondence between the light spot coordinates recognized by the infrared camera and the projection optical machine screen coordinates, the infrared camera and the visible light camera are calibrated with external parameters in advance to obtain the conversion matrix H1 from the infrared camera screen coordinates to the visible light camera screen coordinates; the conversion matrix H1 is the first sub-conversion matrix, and the conversion matrix H1 can be obtained by any of the following two specific steps:

[0050] (1) A spatial coordinate system can be established by the distance between the infrared camera and the visible light camera on the projector, as well as their respective field of view angles. Based on these known hardware parameters, the spatial relative position relationship between the shooting range of the infrared camera and the shooting range of the visible light camera can be determined from the spatial coordinate system, and the conversion matrix H1 can be calculated based on this. For example, the coordinates of the four vertices of the infrared shooting picture in the spatial coordinate system can be obtained based on the field of view angle parameters of the infrared camera. Similarly, the coordinates of the four vertices of the visible light shooting picture in the spatial coordinate system can be obtained based on the known optical parameters. By taking one of the vertices of the infrared shooting picture as the origin of the coordinate system, an infrared shooting picture coordinate system is established, and the coordinates of the visible light shooting picture in the infrared shooting picture coordinate system can be obtained, so that the conversion matrix H1 between the infrared shooting picture coordinate system and the visible light shooting picture coordinate system can be established.

[0051] (2) Before the equipment leaves the factory, the coordinates of the light spot at the same position on the wall in the infrared shooting picture and the visible light shooting picture are measured. According to the coordinates of this light spot in the two shooting pictures, a mapping matrix H1 can be established, such as: Z 可见 =H1*Z 红外 , Z 可见 is the coordinate of the light spot in the visible light shooting picture, Z 红外 is the coordinate of the light spot in the infrared shooting picture.

[0052] Furthermore, it is also necessary to obtain the conversion matrix H2 from the visible light camera screen coordinates to the projection screen coordinates after the projector performs trapezoidal correction. The conversion matrix H2 is the second sub-conversion matrix. When the projector performs trapezoidal correction, it is generally necessary to project feature images such as a chessboard / QR code, which need to be photographed by a visible light camera. The following specific steps can be used to obtain the conversion matrix H2:

[0053] When the host of the projector is moved, the projector may be tilted relative to the screen. At this time, the projector projects a chessboard calibration image. After shooting with the visible light camera, the coordinates of the chessboard corner points are identified and compared with the coordinates of the original points on the chessboard. The conversion matrix H2 from the visible light camera image coordinates to the projection image coordinates is calculated.

[0054] That is Z 原图 =H2*Z 可见 .

[0055] Assuming that the infrared camera recognizes that the coordinate of the light spot in the infrared shooting screen coordinate system is Z0, then the mapping coordinate of Z0 in the projection original image coordinate system is Z1 = H2 × H1 × Z0; Figure 4As shown, the coordinates of the light spot A in the infrared shooting screen coordinate system 100 are calculated by the conversion matrix H1 to obtain A1 in the visible light shooting screen coordinate system 200, and the coordinates of A1 are calculated by the conversion matrix H2 to obtain A2 in the projection original image coordinate system 300; the coordinates of the light spot B are calculated by the conversion matrix H1 to obtain B1 in the visible light shooting screen coordinate system 200, and the coordinates of B1 are calculated by the conversion matrix H2 to obtain B2 in the projection original image coordinate system 300. It can be seen that A is a light spot outside the projection screen 400, and B is a light spot inside the projection screen 400. The projection original image 500 is the image to be projected in the projector. The projection original image 500 in the projection original image coordinate system 300 has been subjected to trapezoidal correction. Although it is trapezoidal, the image after being projected onto the target projection surface will appear as a rectangle; the feature map 600 can be, for example, a checkerboard calibration map.

[0056] Through the transformation matrices H1 and H2, the four vertices C11, C12, C13, and C14 in the projection original image 500 can be respectively converted into C01, C02, C03, and C04 in the infrared shooting screen coordinate system 100. Therefore, the processor can also directly determine whether the light spot is outside the projection screen 400 in the infrared shooting screen coordinate system 100.

[0057] In other embodiments, if the light spot emitted by the remote control is a visible light spot, the captured image corresponds to a visible light captured image. At this time, a calibration image can be projected onto the target projection surface by a projector, and the captured image includes the calibration image. Further, based on the captured image and the calibration image, the main transformation matrix between the captured coordinate system and the projection original image coordinate system can be determined.

[0058] S2: Based on the main transformation matrix and the projection area in the projection original image coordinate system, determine the projection mapping area corresponding to the projection area in the shooting coordinate system;

[0059] like Figure 4 As shown, the area covered by the projection original image 500 in the projection original image coordinate system 300 is the projection area. Combined with the inverse matrix of the main conversion matrix, the corresponding projection mapping area in the shooting coordinate system can be determined, wherein the projection mapping area is the area covered by the projection screen 400 in the infrared shooting screen coordinate system 100. For example, by operating the coordinates of C11, C12, C13, and C14 in the projection original image coordinate system with the inverse matrix of the main conversion matrix, the corresponding coordinates C01, C02, C03, and C04 in the shooting coordinate system can be obtained.

[0060] S3: Calculate the theoretical light spot coordinates in the shooting coordinate system. When the theoretical light spot coordinates are outside the projection mapping area; or when the theoretical light spot coordinates are within the projection mapping area but the light spot cannot be recognized by the camera, instruct the remote control to reduce the brightness of the emitted light spot.

[0061] The theoretical light spot coordinates in the shooting coordinate system may be calculated in any one of the following two ways.

[0062] The first method includes the following steps:

[0063] Acquire a captured image and determine the starting light spot coordinates from the captured image; acquire a first real-time motion parameter of an inertial sensor of a remote controller; and determine theoretical light spot coordinates in a captured coordinate system according to the starting light spot coordinates and the first real-time motion parameter of the inertial sensor of the remote controller, wherein the captured coordinate system is established based on the captured image.

[0064] In a specific example, the remote control emits an infrared laser, and the infrared camera of the projector obtains the infrared shooting picture. After the screen coordinates of the infrared laser spot are identified from the infrared shooting picture, the spot track can be calculated by integration calculation combined with the IMU real-time parameters reported by the remote control through the Bluetooth channel. The IMU real-time parameters include linear acceleration and angular velocity. By integrating the linear acceleration and angular velocity over time, the spatial movement distance of the remote control and the rotation angle around three axes, namely the three attitude angles, can be calculated respectively.

[0065] refer to Figure 5 , the first IMU unit on the remote controller 10 can measure changes in its angle and acceleration. When the IMU in the remote controller 10 recognizes that its spatial position is always changing within a range, it is assumed that the spatial position of the remote controller remains unchanged, and then a corresponding relationship between the moving distance of the light spot in the shooting coordinate system and the change in the attitude angle of the remote controller can be established. In one example, when the spatial position of the remote controller remains unchanged, for example, after integrating the linear acceleration for a preset time, it is found that the moving components of the remote controller in the three-axis direction are all less than a threshold value, then the ratio of the angle change of the remote controller when the light spot moves from K1 to K2 is obtained: k = d1 / θ1. Specifically, the user points the remote control toward the wall in front of the projector and presses the calibration button on the remote control for the first time. At this time, the infrared camera of the projector captures the infrared spot K1 emitted by the remote control to the wall, and records the current horizontal attitude angle 1. The horizontal attitude angle is the flight angle among the three attitude angles. Then the user slightly changes the direction of the remote control and presses the calibration button again. At this time, the infrared camera of the projector captures the infrared spot K2 emitted by the remote control, and records the current horizontal attitude angle 2. The horizontal attitude angle 1 is subtracted from the horizontal attitude angle 2 to obtain θ1. The horizontal offset between K2 and K1 in the infrared shooting picture is d1. In other methods, θ1 can also be the difference between the longitudinal attitude angles when the calibration button is pressed twice, that is, the change in the pitch angle. At this time, d1 is the longitudinal offset of the light spot in the infrared shooting picture when the calibration button is pressed twice.

[0066] Figure 5 The example above only uses the horizontal movement. The ratio k can also be used to calculate the vertical movement of the light spot in the shooting coordinate system. In order to further improve the accuracy, the ratio of the vertical light spot offset to the angle change can also be calibrated and calculated separately by the method in the previous paragraph. Different k values ​​are used for the subsequent vertical and horizontal light spot offset calculations.

[0067] Furthermore, by multiplying the changing attitude angle △θ of the remote control by the ratio k, the change △d of the light spot position can be obtained, that is, △d=k*△θ. For example, d2=k*θ2, d1=k*θ1; from K1 to K2 to K3, each calculation cycle can be a preset value, that is, each cycle can measure the latest coordinates of the light spot in the infrared shooting screen coordinate system, but K1 as the starting light spot coordinate is identified in the shooting screen. In multiple calculation cycles, the remote control moves from K1→K2→K3→K4 in sequence, and the lateral angle changes θ1, θ2, θ3 measured by the remote control IMU are combined with the previously calculated k value, so that the lateral offsets d1, d2, d3 can be known. Then, the lateral offsets such as d1, d2, d3 are accumulated on the basis of the K1 coordinate recorded at the beginning in the infrared shooting screen, and finally the theoretical lateral coordinates of the light spot (such as K2, K3, K4 positions) can be calculated.

[0068] In another specific example, other existing methods of performing track estimation through IMU integration may also be used, which is not limited to the integration method in the above example.

[0069] The second method includes the following steps:

[0070] Acquire a captured image and determine the starting light spot coordinates from the captured image; establish a first spatial coordinate system including the captured image based on the current projection distance of the projector, the first spatial attitude angle of the projector, and the preset viewing angle of the captured image; obtain the starting coordinates and starting orientation of the remote control in the first spatial coordinate system based on the starting light spot coordinates, the distance of the remote control from the wall, and the second spatial attitude angle of the remote control; acquire the second real-time motion parameters of the remote control, and determine the theoretical light spot coordinates in the captured image coordinate system based on the starting coordinates and starting orientation in the first spatial coordinate system and the second real-time motion parameters of the remote control. The captured image coordinate system is established based on the captured image, and the distance of the remote control from the wall refers to the distance between the remote control and the target projection surface.

[0071] In this calculation method, a TOF (Time of Flight) sensor needs to be set on the remote control and the projector respectively. The TOF sensor on the remote control predicts whether the remote control is pointing at the projection screen, thereby controlling whether the laser is turned on or whether the light power is reduced to protect the infrared laser from accidentally hurting the human eye.

[0072] The TOF sensors installed on the remote control and projector can calculate the distance of the two from the wall (the distance from the target projection surface) respectively, and send out two symmetrical beams of light with a known angle to calculate the inclination angle relative to the wall. Figure 6 As shown, the angle φ0 between the two beams of light is known, and the distances L1, L2, and L3 can be calculated based on the time it takes for the light beams to return. Based on these parameters, the inclination angle of the remote controller 10 relative to the target projection surface 30 in the horizontal direction, that is, the flight angle φ1 of the remote controller 10 relative to the target projection surface 30 can be calculated. The distance L3 (the distance from the wall) can be measured by the TOF sensor. Combining the distance L3 and the flight angle φ1, plus the pitch angle and rotation angle measured by the first IMU unit in the remote controller, the distance and orientation of the remote controller 10 relative to the light spot P can be accurately calculated, and then the spatial coordinates of the remote controller 10 in the above-mentioned projector spatial coordinate system can be calculated.

[0073] like Figure 7 As shown, the TOF sensor and the second IMU unit provided on the projector 20, as well as the known field of view angle of the infrared shooting picture 700, can establish a projector coordinate system with the optical center of the projector as the origin. In this coordinate system, the coordinates of the four vertices of the infrared shooting picture 700 can be calculated. Based on the light spot P identified in the infrared shooting picture 700 at the initial time, the initial spatial coordinates and orientation of the remote control 10 in the projector coordinate system can be deduced. Subsequently, the new spatial coordinates and orientation of the remote control in the projector coordinate system are calculated at regular intervals according to the changes in the three attitude angles of the remote control 10 and the distance from the wall, and then the latest position of the light spot P relative to the projection area after the orientation of the remote control changes can be directly calculated by a simple spatial geometry method.

[0074] In the projector coordinate system, the coordinates of the four vertices of the infrared shooting screen 700 can be calculated. According to the calculated relative position of the latest light spot P in the infrared shooting screen 700 in the projector coordinate system, the coordinates of the light spot P in the infrared shooting screen coordinate system can be obtained, that is, the theoretical light spot coordinates in the infrared shooting screen coordinate system can be obtained.

[0075] In this calculation method, the integral calculation process of the light spot track is omitted, which reduces the error generated, thereby improving the calculation accuracy of the theoretical light spot coordinates. That is, the light spot is identified in the infrared shooting picture, and the coordinates of the light spot in the infrared shooting picture coordinate system are bound to the current position and attitude angle (flight angle, pitch angle, rotation angle) of the remote control.

[0076] In the above two processes of calculating the theoretical light spot coordinates in the shooting coordinate system, when the spatial position change of the remote controller exceeds the second threshold, the process returns to the initial step to re-determine the starting light spot coordinates from the shooting screen. Specifically, when the first IMU unit of the remote controller detects that it is in a stationary state, the laser emission of the remote controller can be turned off; when the spatial position change of the remote controller is calculated by the first IMU unit of the remote controller to exceed the second threshold, for example, when the remote controller exceeds 10 cm in any direction of the x, y, and z axes, the track calculation of the light spot coordinates is restarted.

[0077] In another embodiment, a TOF sensor may be used to determine the theoretical light spot coordinates. Specifically, Figure 8 As shown, when the projector is equipped with a TOF sensor, the TOF sensor 15 in the remote controller 10 and the TOF sensor of the projector can use the same band, such as the 940nm band. The TOF sensor 15 in the remote controller 10 can emit a predefined coded signal. When the coded signal is recognized by the TOF sensor in the projector, it means that the remote controller 10 is pointing to the optical screen of the projector, and a signal is sent to the remote controller 10 via Bluetooth to restore its laser power to normal.

[0078] In step S3, the remote controller is specifically instructed to reduce the brightness of the emitted light spot in any of the following two situations: ① when the theoretical light spot coordinates are outside the projection mapping area; ② when the theoretical light spot coordinates are within the projection mapping area, but the light spot cannot be identified by the camera. Among them, when the theoretical light spot coordinates are outside the projection mapping area, the projector transmits a signal through the wireless communication module to instruct the remote controller to reduce the brightness of the emitted light spot. If the theoretical light spot coordinates return to the projection mapping area again, the remote controller is instructed to increase the brightness of the emitted light spot. Specifically, the projector sends a command to the remote controller to instruct the remote controller to restore the normal laser emission power. It should be noted that when the light emitted by the remote controller is infrared light, the camera is an infrared camera; if it is visible light, the camera is a visible light camera.

[0079] In a specific example, when the infrared camera cannot recognize the infrared light spot emitted by the remote control, the light spot track is continuously calculated according to the above two methods of calculating the theoretical light spot coordinates in the shooting coordinate system. If it is found that the theoretical light spot coordinates exceed the projection screen range, an instruction is sent through the Bluetooth channel to the pointing remote control hardware to reduce the laser power or turn off the laser emission.

[0080] The coordinates of the projection image in the infrared shooting image can be calculated based on the conversion matrices H1 and H2 calculated above. Figure 4The coordinates of the vertices are (0,0), (1960,0), (0,1080), and (1960,1080). By substituting these coordinates into the inverse matrix operations of H2 and H1 in turn, the coordinates of these vertices in the coordinate system of the infrared shooting screen can be calculated, which can then be used to determine whether the theoretical spot coordinates calculated by the previous steps are in the projection screen.

[0081] When the infrared camera fails to recognize the infrared light spot and then re-recognizes the light spot coordinates in the infrared shooting picture, the above-mentioned integral calculation error is corrected according to the real coordinates recognized in the infrared shooting picture. For example, when the light spot is re-recognized in the infrared shooting picture, the track of the theoretical light spot coordinates is recalculated with the coordinates of the light spot as the starting coordinates.

[0082] When the track of the calculated theoretical light spot coordinates is within the projection screen, but the infrared camera cannot identify the infrared light spot within the third threshold time (such as within 1s), the remote control is instructed to turn off the laser emission or reduce the emission brightness of the light spot. The third threshold is calculated based on the laser emission frequency, limited to harmless to the human eye; specifically, when the laser emission power is large, the brightness of the emitted laser is high, and the maximum time allowed to irradiate the human eye should be short, otherwise it will damage the human eye; when the calculated theoretical light spot coordinates are within the projection screen, but the infrared camera cannot identify the light spot, it is considered that the light spot may irradiate the eyes of the user standing in the projection screen area, so the timing starts at this time. When this state lasts for more than the third threshold time, the remote control is instructed to reduce the brightness of the emitted light spot to prevent the laser from irradiating the human eye for a long time.

[0083] In some embodiments, after step S3, the exposure time of the camera can be increased and the shooting picture can be obtained, so that when the low-brightness light spot returns to the shooting picture, it can be quickly recognized by the camera; then when the camera recognizes the light spot emitted by the remote control, the remote control is instructed to increase the brightness of the emitted light spot and restore the exposure time of the camera, thereby reducing the impact of ambient light interference on the camera. If the exposure time is always too long, the camera is easily interfered by ambient light. In this embodiment, when the light emitted by the remote control is infrared light, the camera is an infrared camera; if it is visible light, the camera is a visible light camera.

[0084] Through the above steps S1 to S3, the laser in the remote control can be made to work continuously only when it is pointed at the projection screen area, and the laser does not work when it is pointed at other directions. In a further solution, the irradiation angle and brightness of the laser when the remote control is initially started can also be limited. Therefore, before executing step S3, the following steps can also be performed:

[0085] A1: Get the projection distance and pitch angle of the projector, and calculate the initial pitch angle range of the remote control based on the projection distance, pitch angle and the preset remote control usage distance;

[0086] The first IMU unit of the remote control and the second IMU unit of the projector are both calibrated in the horizontal direction. Specifically, they can be calibrated before leaving the factory so that the remote control and the projector are placed on a horizontal plane for IMU calibration.

[0087] like Fig. 9 As shown, based on the default projection distance D and the pitch field angle θ of the projector (i.e., the optical and mechanical field angle of the projector), combined with Fig.10 , according to the preset use distance of the remote controller 10 (generally 2m), calculate the range of the initial pitch angle θk of the remote controller 10, for example, the value range of θk is 0-70°, within which the laser of the remote controller 10 can point to the projection screen of the projector; wherein, the pitch angle θk is the angle between the remote controller and the lines connecting the highest point and the lowest point of the projection screen height h at the preset use distance, and the pitch field angle θt is the angle between the projection angle of the projector in the vertical direction and the lines connecting the highest point and the lowest point of the projection screen height h. θt and θk can be calculated in advance.

[0088] A2: Get the current pitch angle of the remote controller and determine whether the current pitch angle of the remote controller is within the initial pitch angle range;

[0089] The current pitch angle of the remote controller is acquired through the first IMU unit of the remote controller, and it is determined whether the current pitch angle is within the range of the initial pitch angle θk.

[0090] A3: When the current pitch angle of the remote controller is within the initial pitch angle range, the remote controller is driven to emit a light spot at a transmission power lower than a first threshold;

[0091] When it is determined that the current pitch angle is within the range of the initial pitch angle θk, the infrared laser driving the remote control works normally through pulse conduction, but at this time the pulse width (duty cycle) can be controlled to the minimum value that the infrared camera can identify the heat dissipation spot to prevent the laser from being too bright when it is first emitted. The infrared camera on the projector can identify low-power infrared spots by increasing the exposure time.

[0092] A4: Determine whether the light spot emitted by the remote control is recognized by the camera. If so, instruct the remote control to increase the brightness of the emitted light spot.

[0093] In this step, when the infrared camera recognizes the pulsed laser spot, it restores the normal exposure time and notifies the remote control through the Bluetooth channel to increase the laser emission power. At this time, the projector begins to normally recognize the screen coordinates of the spot.

[0094] In a remote control method for a projector disclosed in an implementation of the present invention, a laser light source (infrared light) irradiates a target projection surface (wall / screen, etc.) to scatter a light spot, a camera captures the position of the infrared light spot, calculates the position of the light spot in the camera's shooting coordinate system, and then calculates the position of the light spot in the projection original image coordinate system through the corresponding main transformation matrix; it can further identify whether the light spot leaves the projection screen area, and after leaving the projection screen area, the brightness of the light spot of the remote control can be automatically reduced or turned off to prevent damage to the human eye.

[0095] Embodiment 2 of the present invention discloses a projection system, including a projector and a remote control, wherein the projector includes a processor and a memory, the memory stores a computer program, and the computer program can be read and executed by the processor to implement the remote control method for the projector in embodiment 1.

[0096] The memory is a storage device that stores programs and data executed by the processor in a non-volatile manner. The memory is composed of semiconductor storage elements such as magnetic storage devices, flash read-only memories (ROM), or other types of non-volatile storage devices. The memory may also include random access memories (RAM) that constitute the working area of ​​the processor. The memory stores the control program executed by the processor. The processor may be composed of a single processor or a combination of multiple processing groups. The processor executes the control program to control various parts of the projection device.

[0097] Embodiment 3 of the present invention discloses a storage medium, in which a computer program is stored. The computer program is configured to be executable by a processor to execute the remote control method for a projector in embodiment 1.

[0098] Optionally, the storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0099] The background section of the present invention may contain background information about the problem or environment of the present invention, rather than describing the prior art by others. Therefore, the content contained in the background section is not an admission by the applicant that the prior art is available.

[0100] The above content is a further detailed description of the present invention in combination with specific / preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, without departing from the concept of the present invention, it can also make several substitutions or modifications to these described embodiments, and these substitutions or modifications should be regarded as belonging to the protection scope of the present invention. In the description of this specification, the description of reference terms "an embodiment", "some embodiments", "preferred embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in combination with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, in the absence of mutual contradiction, the technical personnel in this field can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions and modifications can be made herein without departing from the scope defined by the attached claims.

Claims

1. A remote control method for a projector, It is characterized in that The following steps are involved: Get the main transformation matrix between the shooting coordinate system and the projection original image coordinate system; Based on the main transformation matrix and the projection area in the projection original image coordinate system, determining a projection mapping area corresponding to the projection area in the shooting coordinate system; The theoretical light spot coordinates in the shooting coordinate system are calculated, and when the theoretical light spot coordinates are outside the projection mapping area; or when the theoretical light spot coordinates are within the projection mapping area but the light spot cannot be recognized by the camera, the remote control is instructed to reduce the brightness of the emitted light spot.

2. The remote controller control method according to claim 1, It is characterized in that Before calculating the theoretical light spot coordinates in the shooting coordinate system, the following steps are also included: Acquire the projection distance and the pitch angle of the projector, and calculate the initial pitch angle range of the remote control based on the projection distance, the pitch angle of the projector, and a preset remote control usage distance; Acquire the current pitch angle of the remote controller, and determine whether the current pitch angle of the remote controller is within the initial pitch angle range; When the current pitch angle of the remote controller is within the initial pitch angle range, the remote controller is driven to emit a light spot at a transmission power lower than a first threshold.

3. The remote controller control method according to claim 2, It is characterized in that When the current pitch angle of the remote controller is within the initial pitch angle range, the remote controller is driven to emit a light spot with a transmission power lower than a first threshold, and the following steps are also included: It is determined whether the light spot emitted by the remote controller is recognized by the camera, and if so, the remote controller is instructed to increase the brightness of the emitted light spot.

4. The remote controller control method according to claim 1, It is characterized in that The calculating of the theoretical light spot coordinates in the shooting coordinate system comprises: Acquire a shooting picture, determine the starting light spot coordinates from the shooting picture, and acquire a first real-time motion parameter of an inertial sensor in the remote controller; determine theoretical light spot coordinates in the shooting coordinate system according to the starting light spot coordinates and the first real-time motion parameter of the remote controller, wherein the shooting coordinate system is established based on the shooting picture; or, Acquire a captured image and determine the starting light spot coordinates from the captured image; establish a first spatial coordinate system including the captured image based on the current projection distance of the projector, a first spatial attitude angle and a preset viewing angle of the captured image; obtain the starting coordinates and starting orientation of the remote control in the first spatial coordinate system based on the starting light spot coordinates, the distance of the remote control from the wall and the second spatial attitude angle; acquire the second real-time motion parameters of the remote control, and determine the theoretical light spot coordinates in the captured image coordinate system based on the starting coordinates and starting orientation of the remote control in the first spatial coordinate system and the second real-time motion parameters of the remote control. The captured image coordinate system is established based on the captured image, and the distance of the remote control from the wall refers to the distance between the remote control and the target projection surface.

5. The remote controller control method according to claim 4, It is characterized in that The calculating of the theoretical light spot coordinates in the shooting coordinate system further includes: when the spatial position change of the remote controller exceeds a second threshold, re-determining the starting light spot coordinates from the shooting picture.

6. The remote controller control method according to claim 1, It is characterized in that When the theoretical light spot coordinates are outside the projection mapping area, the step of instructing the remote controller to reduce the brightness of the emitted light spot further includes: Increase the exposure time of the camera and obtain the shooting picture; when the camera recognizes the light spot emitted by the remote control, instruct the remote control to increase the brightness of the emitted light spot and restore the exposure time of the camera; or, When the theoretical light spot coordinates are located within the projection mapping area, the remote controller is instructed to increase the brightness of the emitted light spot.

7. The remote controller control method according to claim 1, It is characterized in that The method of obtaining the main conversion matrix between the shooting coordinate system and the projection original image coordinate system includes: obtaining a first sub-conversion matrix preset between the infrared shooting screen and the visible light shooting screen; projecting a feature map onto the target projection surface and obtaining a visible light shooting screen, wherein the visible light shooting screen includes the feature map; obtaining a second sub-conversion matrix based on the coordinates of the feature map in the projection original image and the visible light shooting screen respectively; and obtaining the main conversion matrix based on the first sub-conversion matrix and the second sub-conversion matrix.

8. The remote controller control method according to claim 1, It is characterized in that The shooting coordinate system is an infrared shooting screen coordinate system, and the main transformation matrix is ​​composed of a first sub-transformation matrix and a second sub-transformation matrix. The first sub-transformation matrix is ​​a transformation matrix between the infrared shooting screen coordinate system and the visible light shooting screen coordinate system, and the second sub-transformation matrix is ​​a transformation matrix between the visible light shooting screen coordinate system and the projection original image coordinate system after trapezoidal correction is performed on the projection area.

9. A projection system, It is characterized in that The invention comprises a projector and a remote controller, wherein the projector comprises a processor and a memory, wherein a computer program is stored in the memory, and the computer program can be read and executed by the processor to implement the remote controller control method according to any one of claims 1 to 8.

10. A storage medium, It is characterized in that The storage medium stores a computer program, wherein the computer program is configured to be executable by a processor to execute the remote controller control method according to any one of claims 1 to 8.

Citation Information

Cited By

  • Projection interaction device and method based on invisible light

    CN120786044A

  • Projector control method and device, electronic equipment and storage medium

    CN122053803A