Interaction method for projection picture, projector and storage medium

By calculating the coordinates of the intersection point of the remote control on the projection screen and displaying the cursor, the problem of cumbersome interaction of the projection screen is solved, and the remote control of the projection screen is realized, which improves the operation convenience.

CN118972530APending Publication Date: 2024-11-15深圳市当智科技有限公司
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411041765.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

When the user is performing projection screen interaction, he needs to press the remote control direction button multiple times to make minor adjustments, resulting in high cumbersome operation.

Method used

By obtaining the spatial position and direction information between the projector and the remote control, calculate the coordinates of the intersection point of the remote control on the projection screen, and display the cursor at that location, the remote control's direction and remote control of the projection screen is realized, simplifying interactive operation.

Benefits of technology

It improves the convenience of projection screen interaction, realizes the absolute coordinate positioning touch interaction of the remote control, and simplifies the operation process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118972530A_ABST
    Figure CN118972530A_ABST
Patent Text Reader

Abstract

The invention discloses an interaction method for projecting a picture, a projector and a storage medium, and the method comprises the following steps: obtaining a projection distance of the projector relative to a target projection surface, and a light path angle when the projector projects to the target projection surface, according to the projection distance and the light path angle, determining projection area coordinates of a projection picture on the target projection surface in a projector coordinate system; obtaining the coordinate and the direction of a remote controller in the projector coordinate system; calculating coordinates of an intersection point of the direction of the remote controller on the projection picture in the projector coordinate system by combining the coordinates of the projection area and the coordinates and the direction of the remote controller in the projector coordinate system; and displaying a cursor at a position corresponding to the intersection point coordinate on the projection picture. The directional remote control of the remote controller on the projection picture is realized, so that the convenience of interactive operation is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of projection equipment, and in particular to an interactive method for projecting images, a projector and a storage medium. Background Art

[0002] Users often have to use remote control buttons to interact with the projected image. When moving a selection box in the projected image, or moving an audio or video progress bar, volume bar, brightness bar, or other directional interactive operations, it is generally necessary to press the corresponding direction buttons on the remote control multiple times to make slight adjustments one by one, which makes the interactive operations cumbersome. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes an interactive method for projecting a picture, a projector and a storage medium, which realize the pointing remote control of the projected picture by a remote controller, thereby improving the convenience of interactive operation.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present invention discloses an interactive method for projecting a picture, comprising the following steps:

[0006] Obtaining a projection distance of the projector relative to a target projection surface and an optical path angle when the projector projects onto the target projection surface, and determining the projection area coordinates of the projection image on the target projection surface in the projector coordinate system according to the projection distance and the optical path angle;

[0007] Obtaining the coordinates and orientation of the remote control in the projector coordinate system;

[0008] Calculate the intersection coordinates of the direction of the remote control on the projection screen in the projector coordinate system by combining the projection area coordinates and the coordinates and direction of the remote control in the projector coordinate system;

[0009] A cursor is displayed at a position corresponding to the intersection coordinates on the projection screen.

[0010] Preferably, the acquiring the projection distance of the projector relative to the target projection surface, and the optical path angle when the projector projects onto the target projection surface, and determining the projection area coordinates of the projection picture on the target projection surface in the projector coordinate system according to the projection distance and the optical path angle comprises:

[0011] Obtaining the inclination angle of the projector relative to the target projection surface and the preset optical-mechanical field angle, and obtaining the optical path angle when the projector projects onto the target projection surface according to the optical-mechanical field angle and the inclination angle;

[0012] The projection distance of the projector relative to the target projection surface is obtained, and the projection area coordinates of the projection image on the target projection surface in the projector coordinate system are determined according to the optical path angle and the projection distance.

[0013] Preferably, the projector is provided with a first UWB antenna, the remote control is provided with a second UWB antenna, and the acquiring the coordinates of the remote control in the projector coordinate system comprises:

[0014] respectively acquiring ranging signals from the second UWB antenna through the plurality of the first UWB antennas;

[0015] The coordinates of the remote controller relative to the projector coordinate system are obtained according to the difference between the ranging signals acquired by the first UWB antennas.

[0016] Preferably, the projector is provided with a plurality of first UWB antennas, the remote control is provided with a plurality of second UWB antennas, at least three of the second UWB antennas are arranged in a non-linear manner, and the acquiring the coordinates and orientation of the remote control in the projector coordinate system comprises:

[0017] respectively acquiring, through the plurality of the first UWB antennas, ranging signals from the respective second UWB antennas;

[0018] The orientation of the remote control in the projector coordinate system is obtained according to the difference in the ranging signals obtained by each of the first UWB antennas and a preset antenna arrangement relationship, wherein the antenna arrangement relationship includes a spatial relationship between the front end orientation of the remote control and each of the second UWB antennas.

[0019] Preferably, the step of obtaining the coordinates and orientation of the remote control in the projector coordinate system includes:

[0020] Obtain a first vector in the projector coordinate system and a second vector in the remote control coordinate system between the origin of the projector coordinate system and the origin of the remote control coordinate system;

[0021] Obtaining a first rotation matrix of the projector according to an angle between a first gravity vector in the projector coordinate system and a coordinate axis and a coordinate plane of the projector coordinate system, wherein the first rotation matrix of the projector includes a rotation relationship that makes the z-axis in the projector coordinate system coincide with the first gravity vector;

[0022] Obtaining a first rotation matrix of the remote control according to an angle between a second gravity vector in the remote control coordinate system and a coordinate axis and a coordinate plane of the remote control coordinate system, wherein the first rotation matrix of the remote control includes a rotation relationship that makes the z axis in the remote control coordinate system coincide with the second gravity vector;

[0023] According to the first vector and the first rotation matrix of the projector, a first rotation vector is obtained, and according to the angle between the first rotation vector and the coordinate axis of the projector coordinate system, a second rotation matrix of the projector is obtained; according to the second vector and the second rotation matrix of the projector, a second rotation vector is obtained, and according to the angle between the second rotation vector and the coordinate axis of the remote control coordinate system, a second rotation matrix of the remote control is obtained;

[0024] Obtain a translation matrix according to the distance between the origin of the projector coordinate system and the origin of the remote controller coordinate system of the remote controller, and the angle between the first vector and each coordinate axis of the projector coordinate system;

[0025] According to the operation of the specified coordinate axis of the remote control coordinate system with the first rotation matrix of the remote control, the second rotation matrix of the remote control, the translation matrix, and the inverse matrix of the first rotation matrix of the projector and the second rotation matrix of the projector, the equation of the straight line mapped by the specified coordinate axis in the projector coordinate system is obtained, and the equation of the straight line represents the coordinates and orientation of the remote control in the projector coordinate system.

[0026] Preferably, the projector and the remote controller are both provided with an IMU module, and the first gravity vector and the second gravity vector are determined by the following method:

[0027] Acquire first attitude angle data of an IMU module in the projector, and determine a first gravity vector in the projector coordinate system according to the first attitude angle data;

[0028] Acquire second attitude angle data of the IMU module in the remote controller, and determine a second gravity vector in the remote controller coordinate system according to the second attitude angle data.

[0029] Preferably, combining the projection area coordinates and the coordinates and orientation of the remote control in the projector coordinate system to calculate the intersection coordinates of the orientation of the remote control on the projection screen in the projector coordinate system includes:

[0030] According to the projection area coordinates of the projection picture on the target projection surface in the projector coordinate system, establishing the plane equation of the projection picture in the projector coordinate system;

[0031] The coordinates of the intersection between the direction of the remote control and the projection image in the projector coordinate system are calculated according to the plane equation and the coordinates and direction of the remote control in the projector coordinate system.

[0032] Preferably, after displaying a cursor at a position corresponding to the intersection coordinates on the projection screen, the method further comprises:

[0033] Get the button control instructions of the remote control;

[0034] An operation corresponding to the key control instruction is performed at a position corresponding to the intersection coordinates in the projection image.

[0035] In a second aspect, the present invention discloses a projector, comprising a processor and a memory, wherein the memory stores a computer program, and the computer program can be read and executed by the processor to implement the interactive method for projecting a picture as described in the first aspect.

[0036] In a third aspect, the present invention discloses a storage medium storing a computer program, wherein the computer program is configured to be executable by a processor to execute the interactive method for projecting a picture in the first aspect.

[0037] Compared with the prior art, the beneficial effects of the present invention are: the interactive method, projector and storage medium for projected images proposed in the present invention, wherein by obtaining the projection area coordinates of the projection image on the target projection surface in the projector coordinate system and the coordinates and direction of the remote control in the projector coordinate system, the intersection coordinates of the remote control's direction on the projection image can be determined, and the cursor is further displayed at the intersection coordinates, and corresponding touch operations can be performed accordingly, thereby realizing the remote control of the projection image's direction, and using the remote control on the projector to realize the touch interaction method of absolute coordinate positioning, thereby improving the convenience of interactive operations.

[0038] In a further solution, the projector and remote control each have a built-in UWB antenna, which measures the distance and angle between the two in space through wireless signals, so as to realize the perception of the specific direction and position of the remote control pointing to the projector screen, so as to calculate the coordinates of the intersection between the remote control and the projected screen in the projector coordinate system, and display the cursor at the position pointed by the remote control in the projected screen, thereby realizing wireless air-to-air interactive operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a usage scenario diagram of the projector and remote control;

[0040] Figure 2 is a flow chart of an interactive method for projecting images according to a preferred embodiment of the present invention;

[0041] Figure 3 It is a schematic diagram for determining the coordinates of the projection area of ​​the projection picture in the projector coordinate system;

[0042] Figure 4 is a specific example of the antenna distribution within the projector;

[0043] Figure 5is another specific example of antenna distribution within a projector;

[0044] Figure 6 is a specific example of the antenna distribution in the remote control;

[0045] Figure 7 It is another specific example of the distribution of antenna groups in a projector or remote control;

[0046] Figure 8 It is a relationship diagram between the projector coordinate system, the remote control coordinate system and the target projection surface;

[0047] Fig. 9 is the relationship between the gravity component and each coordinate axis in the remote controller coordinate system;

[0048] Fig.10 It is a schematic diagram of the relationship between the projector coordinate system and the remote control coordinate system after relative rotation. DETAILED DESCRIPTION

[0049] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope and application of the present invention.

[0050] It should be noted that when a component is referred to as being "fixed to" or "disposed on" another component, it can be directly on the other component or indirectly on the other component. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, connection can be used for both fixing and circuit / signal communication.

[0051] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0052] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0053] A preferred embodiment of the present invention discloses an interactive method for projecting a screen, by setting a wireless space positioning module between a remote control and a projector, and obtaining spatial (distance and direction) information based on the wireless space positioning module on both the remote control and the projector to determine the pointing position of the remote control on the projected screen.

[0054] The technology type of wireless spatial positioning can be UWB technology, Bluetooth BLE technology, or other wireless technologies that support distance measurement and angle of arrival measurement. The following uses UWB technology to illustrate how to calculate the distance and azimuth angle between the remote control and the projector. For example, the TOA (Time of Arrival) ranging method, the AOA (Angle of Arrival) positioning method, or the PDOA (Phase Difference of Arrival) angle measurement method can be used.

[0055] Among them, the TOA ranging method measures the propagation time from the transmitter to the receiver and multiplies it by the speed of the electromagnetic wave to get the distance. This method requires strict clock synchronization between the transmitter and the receiver, otherwise there will be time deviation, resulting in ranging deviation. The TW-TOF (Two Way Time of Flight) method based on the TOA ranging method is more commonly used. For example, to measure the distance between A and B, first A is used as the transmitter and B is used as the receiver - A sends and B receives, then B is used as the transmitter and A is used as the receiver - B sends and A receives, where the time difference between A sending and receiving is ta, and the time difference between B sending and receiving is tb. The time difference between ta and tb is the time it takes for the electromagnetic wave to go back and forth between A and B. Based on ta and tb, the distance between A and B can be calculated. This method does not require A and B clock synchronization, but in order to calculate the accuracy of the absolute value of time, high clock accuracy is required, otherwise it will cause deviation. In order to reduce the ranging error caused by clock deviation, measurements in both forward and reverse directions can be used. For example, first measure in the forward direction and then in the reverse direction. That is, B sends and A receives, and A responds and then sends and B receives. By measuring in the forward and reverse directions multiple times and averaging them, the impact of clock deviation can be reduced and the accuracy of ranging can be improved.

[0056] The AOA positioning method can measure the incident angle of electromagnetic waves from an unknown point to a reference point, which is usually obtained by the receiver detecting the source direction of the signal energy peak through the antenna array, or by the antenna array measuring the phase difference of the signal.

[0057] The PDOA angle measurement method is to set up two antenna receiving ends in a two-dimensional space, where the distance d between the two antenna receiving ends is known. Based on measuring the phase difference of the received signals at the two antenna receiving ends, the time difference of the received signals can be calculated to calculate the arrival angle of the transmitting end. The specific formula is as follows:

[0058] p=d·sinθ p

[0059]

[0060] Where C and D represent two receiving antennas. When C and D are in the remote control, the transmitting antenna can be set in the projector. When C and D are in the projector, the transmitting antenna is set in the remote control. The transmitting antenna transmits a wireless signal with a wavelength of λ. p is the distance difference between the transmitting antenna and C and D, and d is the distance between the receiving antennas C and D, which is a known value; α p It represents the arrival phase difference of the wireless signal received by the two receiving antennas C and D from the transmitting antenna, and λ is the wavelength of the wireless signal of the transmitting antenna. p It is the angle between the transmitting antenna and the line connecting the receiving antennas C and D, and is used to characterize the angle between the projector and the axis of the remote control.

[0061] Among them, the PDOA angle measurement method requires that the two receiving antennas are very close to each other; further, the optimal spacing between the two receiving antennas is half the wavelength, that is, λ / 2. The optimal antenna spacing refers to the spacing that maximizes the arrival phase difference without generating phase ambiguity. The basic principle of the PDOA angle measurement method is to convert the arrival angle θ p Converted to arrival phase difference α p , the arrival angle is estimated indirectly by measuring the arrival phase difference. p Convert to alpha p The larger the magnification of , the higher the estimation accuracy. When the antenna spacing is the optimal spacing, θ p ∈[-π / 2,π / 2], θ p With α p There is a one-to-one correspondence. When the antenna spacing is smaller than the optimal spacing, the estimation accuracy is lost compared to the optimal antenna spacing. When the antenna spacing is larger than the optimal spacing, there is a phase ambiguity phenomenon, that is, θ p With α p The relationship between α does not satisfy a one-to-one correspondence, but a many-to-one relationship. p It is impossible to uniquely determine θ p This means that for θ p The estimated range has narrowed.

[0062] In summary, the distance between two antenna arrays in three-dimensional space and the angle of arrival between the mutual signals can be calculated through TOA distance measurement and PDOA angle measurement. In addition to the methods given here, there are other methods to measure the pointing point of the remote control on the projected image, which are not listed here one by one.

[0063] like Figure 1As shown, it is a usage scenario diagram of the projector 10 and the remote control 20, wherein the optical center of the projector 10 is marked as O, and the four vertices of the projection surface projected by the projector 10 on the target projection surface are P1, P2, P3, and P4; the origin directly in front of the remote control 20 is pointed to as O′, and the intersection point of the remote control 20 pointing to the target projection surface is P′. To realize that the remote control 20 points to the remote control projection screen, it is necessary to calculate the absolute coordinates of the projection screen pointed by the remote control 20, which is converted into a mathematical problem, that is, it is necessary to find the three-dimensional coordinates of the intersection point P′ of the O′P′ straight line and the projection surface in the projector coordinate system O. According to the angle measurement and distance measurement principles of the aforementioned TOA ranging method and PDOA angle measurement method, the antenna layout of the projector 10 and the remote control 20, as well as the mathematical relationship between the antenna and the three-dimensional coordinate system are set.

[0064] like Figure 2 As shown, the interactive method for projecting pictures in a preferred embodiment of the present invention comprises the following steps:

[0065] S1: Obtain the projection distance of the projector relative to the target projection surface, and the optical path angle when the projector projects onto the target projection surface, and determine the projection area coordinates of the projection image on the target projection surface in the projector coordinate system according to the projection distance and the optical path angle;

[0066] Wherein, step S1 specifically includes:

[0067] S11: Obtaining the inclination angle of the projector relative to the target projection surface and the preset optical-mechanical field angle, and obtaining the optical path angle when the projector projects onto the target projection surface according to the optical-mechanical field angle and the inclination angle;

[0068] S12: Obtaining the projection distance of the projector relative to the target projection surface, and determining the projection area coordinates of the projection image on the target projection surface in the projector coordinate system according to the optical path angle and the projection distance.

[0069] In a specific example, the coordinates of the four vertices P1, P2, P3, and P4 of the projection surface in the coordinate system O of the projector can be calculated by the optomechanical FOV (field of view) parameters and the projector's automatic trapezoidal algorithm sensor (camera / ToF / gyroscope), where the coordinate system O is a coordinate system with the optomechanical optical center of the projector as the origin, the y-axis in the coordinate system O coincides with the optomechanical optical axis of the projector, and the Oxy plane is parallel to the horizontal plane of the projector body. Consider the case where the projector is projecting the target projection surface, such as Figure 3 As shown, the horizontal angle α of the projector optical machine FOV is known F , pitch angle β F , the optical machine is an upward biased optical machine, that is, ∠P5OP6=β F , ∠P4OP5=α F / 2=∠P3OP5. The projection distance of the optical machine is obtained by the distance measuring sensor TOF, which is k, that is, OP5=k, and the surface equation of the projection surface is y=k. The coordinates of the four vertices in the coordinate system O are: P1(-k*tan(α F / 2),k,k*tanβ F )、P2(k*tan(α F / 2),k,k*tanβ F )、P3(k*tan(α F / 2),k,0)、P4(-k*tan(α F / 2),k,0). After the projector performs automatic trapezoidal correction, the yaw angle / roll angle / pitch angle centered on the optical axis can be calculated according to the automatic trapezoidal algorithm sensor (camera / ToF / gyroscope). Combined with the above, the coordinates of the four vertices in the coordinate system O can be calculated.

[0070] S2: Get the coordinates and orientation of the remote control in the projector coordinate system;

[0071] In some embodiments, a first UWB antenna is provided in the projector and a second UWB antenna is provided in the remote control. This step specifically includes: obtaining ranging signals from the second UWB antenna respectively through multiple first UWB antennas; and obtaining the coordinates of the remote control relative to the projector coordinate system based on the differences in the ranging signals obtained by each first UWB antenna.

[0072] like Figure 4 As shown, it is an antenna distribution in a projector in a specific example, through which the signal incident angle can be measured in the projector coordinate system, wherein antennas E, F, and G are antennas in the projector respectively, the position of antenna F is the origin of the projector coordinate system, the line connecting antenna E and antenna F is perpendicular to the horizontal plane of the projector body, the line connecting antenna F and antenna G is parallel to the optical axis direction (the direction of the central projection light path), and antenna H is the antenna of the transmitting end, for example, the antenna of the remote control.

[0073] The optimal spacing between antennas is λ / 2, and the antenna spacing is in centimeters. When calculating the distance and angle between the transmitting source and the receiving antenna, the antenna spacing can be approximately regarded as 0. Therefore, the distances between antennas E, F, and G and antenna H can be regarded as the distances between the origin (antenna F) and antenna H. The angle between the line connecting antennas E and F and antenna H, as well as the angle between the line connecting antennas F and G and antenna H, can be regarded as the angle between the coordinate axis passing through the origin (antenna F) and antenna H. The distance between antennas H and F is obtained by the TOA ranging method. The angle ∠HFG between HF and FG is represented by ω1 and can be obtained by the PDOA angle measurement method. The angle ∠EFH between HF and EF is obtained by the PDOA angle measurement method. The projection Hˊ of antenna H on the xy plane, the angle between HF and HˊF θ1=90°-∠EFH, according to the spatial three cosine theorem, cosω1=cosθ1*cosФ1, the angle Ф1 between HˊF and FG can be obtained. Therefore, in the projector coordinate system, the y-axis is parallel to the optical axis, the xy plane is parallel to the horizontal plane of the projector body, and the z-axis is perpendicular to the xy plane. The angles between the transmitting antenna H and the x, y, and z axes, as well as the angles between the projection of antenna H on the xy / yz / xz plane and the x / y / z axes can be calculated by the PDOA angle measurement method.

[0074] like Figure 5 As shown, in another specific example, with Figure 4 What is different from the example shown is that the line connecting antenna F and antenna G in the projector is distributed perpendicular to the optical axis direction (the direction of the central projection light path). Through this antenna distribution, the PDOA angle measurement method can also be used to calculate the angle between the antenna H at the transmitting end and the x, y, and z axes, as well as the angle between the projection of antenna H in the xy / yz / xz plane and the x / y / z axes.

[0075] like Figure 6 The figure shows the antenna distribution in a remote control in a specific example. Antennas I, J, and K are antennas installed in the remote control, where the line connecting antenna I and antenna J is perpendicular to the horizontal plane of the remote control, and the line connecting antenna J and antenna K is parallel to the directional distribution of the remote control. Antenna Q is the antenna at the transmitting end, such as the antenna of a projector. The position of antenna J is the origin of the remote control coordinate system. The angle ∠QJK between QJ and JK is represented by ω2, the angle between QJ and QˊJ is represented by θ2, and the angle between QˊJ and JK is represented by Ф2. ω1 and Ф2 can be obtained based on the same principle as the calculation of ω1 and Ф1 mentioned above. By Figure 6For the antenna design shown, the angles between the transmitting antenna Q and the x, y, z axes, and the angles between the projection of the antenna Q on the xy / yz / xz plane and the x / y / z axes can be calculated by the PDOA angle measurement method. In another specific example, the line connecting the antenna J and the antenna K is perpendicular to the directional distribution of the remote control. Based on this antenna distribution, the angles between the transmitting antenna Q and the x, y, z axes, and the angles between the projection of the antenna Q on the xy / yz / xz plane and the x / y / z axes can also be calculated by the PDOA angle measurement method.

[0076] In other embodiments, the projector is provided with a plurality of first UWB antennas, the remote control is provided with a plurality of second UWB antennas, and at least three second UWB antennas are arranged in a non-linear manner. This step specifically includes: obtaining ranging signals from each second UWB antenna respectively through the plurality of first UWB antennas; obtaining the orientation of the remote control in the projector coordinate system according to the difference in ranging signals obtained by each first UWB antenna and the preset antenna arrangement relationship, and the antenna arrangement relationship includes the spatial relationship between the front orientation of the remote control and each second UWB antenna. For example, the plurality of first UWB antennas are arranged along the front and back, left and right, and up and down directions of the projector. According to the wireless signal transmitted by a second UWB antenna on the remote control, the distance from the second UWB antenna to each first UWB antenna can be calculated based on the phase difference principle of the wireless signal and the distance between each first UWB antenna, so that the coordinates of the second UWB antenna can be determined in the established projector coordinate system.

[0077] Among them, considering the directivity of the angle measurement of the wireless signal antenna, in a further specific example, an array is used to measure the incident angle of the incident source pointing to different planes, such as Figure 7 As shown, it can be an antenna in a projector or an antenna in a remote control, wherein antenna M measures the incident signal in the lower half of the xy plane, and antenna group S measures the upper half of the xy plane; antenna group N measures the incident signal in the front half of the xz plane, and antenna group T measures the rear half of the xz plane. In order to measure more accurately, the angle relationship under a smaller antenna FOV (field of view) angle can be measured by adding more antenna groups.

[0078] Based on the above, Figure 8As shown, coordinate system O is the projector coordinate system, the y-axis in coordinate system O coincides with the optical axis of the projector, and the Oxy plane is parallel to the horizontal plane of the projector body; O′ coordinate system is the remote control coordinate system, and the y-axis in O′ coordinate system coincides with the pointing direction of the remote control. By setting multiple antennas in the projector and remote control respectively, based on the above positioning principle, the distance of OO′ is calculated according to the TOA ranging method, and the angle of OO′ relative to each axis of coordinate system O′ and each axis of coordinate system O can be calculated by the PDOA positioning method, so that the coordinates of O′ in coordinate system O and the linear equation of the xyz coordinate axis of O′ in coordinate system O can be calculated. By describing the three-dimensional spatial relationship between the antenna and the projector / remote control, the three-dimensional spatial coordinate system relationship of the projector, remote control, and target projection surface can be further obtained. Among them, in order to simplify the calculation, the center point of the projector antenna group can be set to coincide with the optical center.

[0079] In a specific example, by obtaining the rotation matrix and translation matrix from the coordinate system O' of the remote control to the coordinate system O of the projector, the straight line equation of O'P' in the coordinate system O can be calculated, and the coordinates and orientation of the remote control in the projector coordinate system can be obtained according to the straight line equation, which specifically includes the following steps:

[0080] S21: Obtain a first vector in the projector coordinate system and a second vector in the remote control coordinate system between the origin of the projector coordinate system and the origin of the remote control coordinate system;

[0081] Get the first vector in the projector's coordinate system O The second vector in the remote control's coordinate system O' Among them, α, β, and γ are the angles between the straight line OO' and the three axes of the coordinate system O: the angle between OO' and the x-axis is α, the angle between OO' and the y-axis is β, and the angle between OO' and the z-axis is γ; α', β', and γ' are the angles between the straight line OO' and the three axes of the coordinate system O': the angle between OO' and the x-axis is α', the angle between OO' and the y-axis is β', and the angle between OO' and the z-axis is γ'. According to the principle of the PDOA angle measurement method, α, β, γ and α', β', and γ' can be measured.

[0082] S22: obtaining a first rotation matrix of the projector according to an angle between a first gravity vector in the projector coordinate system and a coordinate axis and a coordinate plane of the projector coordinate system, wherein the first rotation matrix of the projector includes a rotation relationship for aligning a z-axis in the projector coordinate system with the first gravity vector;

[0083] The gravity g vector is used to match the coordinate system O' of the remote control and the coordinate system O of the projector to the same vertical direction. The remote control and the projector have IMU modules (inertial measurement modules) that can measure gravity acceleration. In one embodiment, both the projector and the remote control are equipped with IMU modules, and the first gravity vector and the second gravity vector can be determined by the following method:

[0084] The first attitude angle data of the IMU module in the projector is obtained, and the first gravity vector is determined in the projector coordinate system according to the first attitude angle data; the second attitude angle data of the IMU module in the remote control is obtained, and the second gravity vector is determined in the remote control coordinate system according to the second attitude angle data. The x-axis or y-axis of the projector coordinate system is toward the front of the projector, that is, the direction of the side where the optical machine light outlet is located; the z-axis is toward the top of the projector, so the first attitude angle data measured by the IMU module in the projector can directly reflect the direction of the projector gravity in the projector coordinate system, so that the first gravity vector can be established, which can be a unit vector, and the first attitude angle data includes the yaw angle, roll angle, and pitch angle information of the projector. Similarly, the x-axis or y-axis of the remote control coordinate system is toward the front of the remote control, and the z-axis is toward the top of the remote control. The second attitude angle data measured by the IMU module in the remote control can directly reflect the direction of the gravity of the remote control in the remote control coordinate system, so that the second gravity vector can be established, which can be a unit vector, and the second attitude angle data includes the yaw angle, roll angle, and pitch angle information of the remote control.

[0085] In the coordinate system O of the projector, the components of gravity g on the x, y, and z axes and the projection component of g on the yz plane are known. yz , we can get g yz Angle with z axis Ф yz , g and g yz The angle θ yz The coordinate system O rotates around the x-axis by Ф yz , get the rotation matrix R x (Ф yz ), and then rotate θ around the y axis yz , get the rotation matrix R y (θ yz ); after two rotations, the z axis of the coordinate system O coincides with the g axis. In this specific example, the angle between the first gravity vector in the projector coordinate system and the coordinate axis of the projector coordinate system includes Ф yz , the angle between the first gravity vector in the projector coordinate system and the coordinate plane includes θ yz , the first rotation matrix of the projector includes R x (Ф yz ) and R y (θ yz ).

[0086] S23: obtaining a first rotation matrix of the remote control according to an angle between the second gravity vector in the remote control coordinate system and the coordinate axis and the coordinate plane of the remote control coordinate system, wherein the first rotation matrix of the remote control includes a rotation relationship that makes the z axis in the remote control coordinate system coincide with the second gravity vector;

[0087] like Fig. 9 In the coordinate system O′ of the remote controller, the components of gravity g on the x, y, and z axes and the projection component of g on the yz plane are known. yz ', we can get g yz 'Angle Ф with z axis yz ′, g and g yz 'The angle θ yz ′; coordinate system O′ first rotates around the x-axis by Ф yz ′, get the rotation matrix R' x (Ф yz ′), and then rotate θ around the y axis yz ', get the rotation matrix R' y (θ yz '); After two rotations, the z axis of the coordinate system O' coincides with g. In this specific example, the angle between the second gravity vector in the remote control coordinate system and the coordinate axis of the remote control coordinate system includes Ф yz ′, the angle between the second gravity vector in the remote control coordinate system and the coordinate plane includes θ yz ', the first rotation matrix of the remote control includes R' x (Ф yz ′) and R' y (θ yz ').

[0088] S24: obtaining a first rotation vector according to the first vector and the first rotation matrix of the projector, and obtaining a second rotation matrix of the projector according to the angle between the first rotation vector and the coordinate axis of the projector coordinate system; obtaining a second rotation vector according to the second vector and the second rotation matrix of the projector, and obtaining a second rotation matrix of the remote control according to the angle between the second rotation vector and the coordinate axis of the remote control coordinate system;

[0089] like Fig.10 , after the rotation of coordinate system O and coordinate system O′, the z axis coincides with g, and the xy planes of the two are parallel. After the vector undergoes the same two rotation matrix changes, the angle between it and the xyz axis of each coordinate system becomes α g , β g , γ g With α' g , β' g , γ' g ; Calculate the angle θ between OO′ and the projection of OO′ on the xy plane in the coordinate system O g =|90°-γg |, through the three cosine theorem of space angle cosα g = cosθ g *cosФ g The angle Φ between the projection of OO′ on the xy plane of the coordinate system O and the x-axis can be obtained g Similarly, the angle Φ' between the projection of OO' on the xy plane of the coordinate system O' and the x-axis can be obtained g The coordinate system O is then rotated around the z axis by Ф g , the rotation matrix is ​​R z (Ф g ), let the x-axis be The projections of the vectors on the xy plane are aligned; the coordinate system O′ is also rotated around the z axis by Ф' g , the rotation matrix R' z (Ф' g ), let the x-axis be The projection of the vector on the xy plane is aligned. In this specific example, the angle between the first rotation vector and the coordinate axis of the projector coordinate system includes α g , β g , γ g The angle between the second rotation vector and the coordinate axis of the remote control coordinate system includes α' g , β' g , γ' g , the second rotation matrix of the projector includes R z (Ф g ), the remote control second rotation matrix includes R' z (Ф' g ).

[0090] S25: Obtain a translation matrix according to the distance between the origin of the projector coordinate system and the origin of the remote controller coordinate system of the remote controller, and the angle between the first vector and each coordinate axis of the projector coordinate system;

[0091] The coordinates of O' in the coordinate system O have been obtained above. The distance OO' between the coordinate system O and the coordinate system O' represents the distance between the projector antenna group and the remote control antenna group, which is set to L. L can be calculated according to the TOA ranging method. The first vector in the projector coordinate system between the origin of the projector coordinate system and the origin of the remote control coordinate system is The calculated translation matrix of the origin of coordinate system O′ relative to the origin of coordinate system O is: {Lcosα, Lcosβ, Lcosγ}.

[0092] S26: According to the operation of the designated coordinate axis of the remote control coordinate system with the first rotation matrix of the remote control, the second rotation matrix of the remote control, the translation matrix, and the inverse matrix of the first rotation matrix of the projector and the second rotation matrix of the projector, the equation of the straight line mapped by the designated coordinate axis in the projector coordinate system is obtained, and the equation of the straight line represents the coordinates and orientation of the remote control in the projector coordinate system.

[0093] The designated coordinate axis of the remote control coordinate system is a coordinate axis along the direction pointed by the remote control, such as the x-axis or the y-axis. In this specific example, the direction of the remote control is set along the y-axis, that is, the designated coordinate axis is the y-axis.

[0094] Combining the above rotation transformation process, it can be obtained that after the following rotation transformations, the xyz axes of coordinate system O′ are aligned parallel to coordinate system O in space: coordinate system O′ is first rotated around the x-axis by Ф yz ′, get the rotation matrix R' x (Ф yz ′), and then rotate θ around the y axis yz ', get the rotation matrix R' y (θ yz '), aligned with the gravity g vector; then rotate around the z axis Ф' g , get the rotation matrix R' z (Ф' g ), let the x-axis be Align the projections of the vectors on the xy plane; then rotate the coordinate system O′ in the opposite direction around the z axis. g , that is, R z (Ф g ) Then rotate θ in the opposite direction around the y axis yz , that is, R y (θ yz ) Then rotate Φ in the opposite direction around the x-axis yz , that is, R x (Ф yz ) Among them, the rotation matrix of the entire rotation process is:

[0095]

[0096] The matrix transformation from coordinate system O′ to coordinate system O can be expressed by the following rotation + translation:

[0097]

[0098] Select a point (0,1,0) on the y-axis of coordinate system O′, calculate the coordinates of (0,1,0) in coordinate system O, and represent it with A(a,b,c). Given the coordinates of A and O′ in coordinate system O, we can get the equation of the line of O′ y-axis in coordinate system O. This equation can represent the coordinates and direction of the remote control in coordinate system O.

[0099] Among them, the order of the above steps S1 and S2 can be executed in the order of S1→S2, or in the order of S2→S1, or steps S1 and S2 can be executed simultaneously, and then S3 is further calculated by combining the parameters obtained in steps S1 and S2.

[0100] S3: Calculate the intersection coordinates of the remote control's direction on the projection screen in the projector coordinate system by combining the projection area coordinates and the remote control's coordinates and direction in the projector coordinate system;

[0101] Among them, this step specifically includes:

[0102] S31: establishing a plane equation of the projection picture in the projector coordinate system according to the projection area coordinates of the projection picture on the target projection surface in the projector coordinate system;

[0103] S32: Calculate the coordinates of the intersection between the direction of the remote control and the projection image in the projector coordinate system according to the plane equation and the coordinates and direction of the remote control in the projector coordinate system.

[0104] In a specific example, Figure 8 As shown, P' is the intersection of the y-axis (the axis in the length direction of the remote control) of the coordinate system O' and the projection screen. Therefore, calculating the intersection coordinates of the remote control's pointing direction on the projection screen in the projector coordinate system is to calculate the coordinates of P' in the coordinate system O. According to the specific example in step S2, the linear equation of the O'y-axis in the coordinate system O has been obtained, and further, the surface equation of the intersection point P' formed by the O'y-axis and the four vertices P1, P2, P3, and P4 of the projection surface of the coordinate system O can be obtained. Further, by converting the coordinates of P' to the pixel coordinate system of the optical machine screen, the pointing direction point of the remote control can be obtained in the projection screen.

[0105] In another specific example, after knowing the spatial coordinates of P1, P2, P3, and P4 in coordinate system O, a plane equation can be established based on the coordinates of these four points; and after knowing the distance L between OO′ and measuring α, β, γ and α′, β′, γ′ according to the principle of PDOA angle measurement, the coordinates of O′ in coordinate system O and the straight line equation of the y′ axis in coordinate system O′ in coordinate system O can be calculated; based on the intersection of the straight line equation and the plane equation, the coordinates of P′ in coordinate system O can be obtained.

[0106] S4: Displaying a cursor at a position corresponding to the intersection coordinates on the projection screen.

[0107] The cursor is, for example, a circular cursor, a mouse-shaped cursor, etc., so that the projection screen projected by the projector can display an indicating cursor of a corresponding pattern. Further, in order to realize the interaction of the cursor position, after step S4, it includes: obtaining a key control instruction of the remote controller; and executing the operation of the corresponding key control instruction at the position corresponding to the intersection coordinates in the projection screen.

[0108] In a specific example, according to the relative position coordinates of P' in the plane where P1, P2, P3, and P4 are located, its relative position in the pixel coordinate system of the optical machine screen (such as a 1920*1080 plane) can be obtained, and then the pointing direction point of the remote control can be obtained in the projected screen, and the point can be displayed as a red dot, pointer, etc. in the projected screen. Further, when the user presses a designated button on the remote control, click, slide, etc. are performed on the position of P' in the optical machine screen.

[0109] Embodiment 2 of the present invention discloses a projector, including a processor and a memory, wherein a computer program is stored in the memory, and the computer program can be read and executed by the processor to implement the interactive method for projecting a picture in embodiment 1.

[0110] The memory is a storage device that stores programs and data executed by the processor in a non-volatile manner. The memory is composed of semiconductor storage elements such as magnetic storage devices, flash read-only memories (ROM), or other types of non-volatile storage devices. The memory may also include random access memories (RAM) that constitute the working area of ​​the processor. The memory stores the control program executed by the processor. The processor may be composed of a single processor or a combination of multiple processing groups. The processor executes the control program to control various parts of the projection device.

[0111] Embodiment 3 of the present invention discloses a storage medium, in which a computer program is stored. The computer program is configured to be executable by a processor to execute the interactive method for projecting a picture in embodiment 1.

[0112] Optionally, the storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0113] The background section of the present invention may contain background information about the problem or environment of the present invention, rather than describing the prior art by others. Therefore, the content contained in the background section is not an admission by the applicant that the prior art is available.

[0114] The above content is a further detailed description of the present invention in combination with specific / preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, without departing from the concept of the present invention, it can also make several substitutions or modifications to these described embodiments, and these substitutions or modifications should be regarded as belonging to the protection scope of the present invention. In the description of this specification, the description of reference terms "an embodiment", "some embodiments", "preferred embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in combination with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, in the absence of mutual contradiction, the technical personnel in this field can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions and modifications can be made herein without departing from the scope defined by the attached claims.

Claims

1. An interactive method for projecting a picture, characterized in that: The following steps are involved: Obtaining a projection distance of the projector relative to a target projection surface and an optical path angle when the projector projects onto the target projection surface, and determining the projection area coordinates of the projection image on the target projection surface in the projector coordinate system according to the projection distance and the optical path angle; Obtaining the coordinates and orientation of the remote control in the projector coordinate system; Calculate the intersection coordinates of the direction of the remote control on the projection screen in the projector coordinate system by combining the projection area coordinates and the coordinates and direction of the remote control in the projector coordinate system; A cursor is displayed at a position corresponding to the intersection coordinates on the projection screen.

2. The interactive method for projecting images according to claim 1, characterized in that: The obtaining of the projection distance of the projector relative to the target projection surface and the optical path angle when the projector projects onto the target projection surface, and determining the projection area coordinates of the projection image on the target projection surface in the projector coordinate system according to the projection distance and the optical path angle comprises: Obtaining the inclination angle of the projector relative to the target projection surface and the preset optical-mechanical field angle, and obtaining the optical path angle when the projector projects onto the target projection surface according to the optical-mechanical field angle and the inclination angle; The projection distance of the projector relative to the target projection surface is obtained, and the projection area coordinates of the projection image on the target projection surface in the projector coordinate system are determined according to the optical path angle and the projection distance.

3. The interactive method for projecting images according to claim 1, characterized in that: The projector is provided with a first UWB antenna, the remote controller is provided with a second UWB antenna, and obtaining the coordinates of the remote controller in the projector coordinate system includes: respectively acquiring ranging signals from the second UWB antenna through the plurality of the first UWB antennas; The coordinates of the remote controller relative to the projector coordinate system are obtained according to the difference between the ranging signals acquired by the first UWB antennas.

4. The interactive method for projecting images according to claim 1, characterized in that: The projector is provided with a plurality of first UWB antennas, the remote control is provided with a plurality of second UWB antennas, at least three of the second UWB antennas are arranged in a non-linear manner, and obtaining the coordinates and orientation of the remote control in the projector coordinate system includes: respectively acquiring, through the plurality of the first UWB antennas, ranging signals from the respective second UWB antennas; The orientation of the remote control in the projector coordinate system is obtained according to the difference in the ranging signals obtained by each of the first UWB antennas and a preset antenna arrangement relationship, wherein the antenna arrangement relationship includes a spatial relationship between the front end orientation of the remote control and each of the second UWB antennas.

5. The interactive method for projecting images according to claim 1, characterized in that: The obtaining of the coordinates and orientation of the remote control in the projector coordinate system includes: Obtain a first vector in the projector coordinate system and a second vector in the remote control coordinate system between the origin of the projector coordinate system and the origin of the remote control coordinate system; Obtaining a first rotation matrix of the projector according to an angle between a first gravity vector in the projector coordinate system and a coordinate axis and a coordinate plane of the projector coordinate system, wherein the first rotation matrix of the projector includes a rotation relationship that makes the z-axis in the projector coordinate system coincide with the first gravity vector; Obtaining a first rotation matrix of the remote control according to an angle between a second gravity vector in the remote control coordinate system and a coordinate axis and a coordinate plane of the remote control coordinate system, wherein the first rotation matrix of the remote control includes a rotation relationship that makes the z axis in the remote control coordinate system coincide with the second gravity vector; According to the first vector and the first rotation matrix of the projector, a first rotation vector is obtained, and according to the angle between the first rotation vector and the coordinate axis of the projector coordinate system, a second rotation matrix of the projector is obtained; according to the second vector and the second rotation matrix of the projector, a second rotation vector is obtained, and according to the angle between the second rotation vector and the coordinate axis of the remote control coordinate system, a second rotation matrix of the remote control is obtained; Obtain a translation matrix according to the distance between the origin of the projector coordinate system and the origin of the remote controller coordinate system of the remote controller, and the angle between the first vector and each coordinate axis of the projector coordinate system; According to the operation of the specified coordinate axis of the remote control coordinate system with the first rotation matrix of the remote control, the second rotation matrix of the remote control, the translation matrix, and the inverse matrix of the first rotation matrix of the projector and the second rotation matrix of the projector, the equation of the straight line mapped by the specified coordinate axis in the projector coordinate system is obtained, and the equation of the straight line represents the coordinates and orientation of the remote control in the projector coordinate system.

6. The interactive method for projecting images according to claim 5, characterized in that: The projector and the remote controller are both provided with an IMU module, and the first gravity vector and the second gravity vector are determined in the following manner: Acquire first attitude angle data of an IMU module in the projector, and determine a first gravity vector in the projector coordinate system according to the first attitude angle data; Acquire second attitude angle data of the IMU module in the remote controller, and determine a second gravity vector in the remote controller coordinate system according to the second attitude angle data.

7. The interactive method for projecting images according to claim 1, characterized in that: The step of combining the projection area coordinates and the coordinates and orientation of the remote controller in the projector coordinate system to calculate the intersection coordinates of the orientation of the remote controller on the projection screen in the projector coordinate system includes: According to the projection area coordinates of the projection picture on the target projection surface in the projector coordinate system, establishing the plane equation of the projection picture in the projector coordinate system; The coordinates of the intersection between the direction of the remote control and the projection image in the projector coordinate system are calculated according to the plane equation and the coordinates and direction of the remote control in the projector coordinate system.

8. The interactive method for projecting images according to claim 1, characterized in that: After displaying a cursor at a position corresponding to the intersection coordinates on the projection screen, the method further comprises: Get the button control instructions of the remote control; An operation corresponding to the key control instruction is performed at a position corresponding to the intersection coordinates in the projection image.

9. A projector, characterized in that: It comprises a processor and a memory, wherein a computer program is stored in the memory, and the computer program can be read by the processor to be executed to implement the interactive method for projecting pictures as described in any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to be executable by a processor to execute the interactive method for projecting images as described in any one of claims 1 to 8.

Citation Information

Cited By

  • Vehicle-mounted screen pointing control system and method based on UWB and IMU

    CN121691780A