Interaction method for projection picture, projector and storage medium

Abstract

The invention discloses an interaction method for a projection picture, a projector and a storage medium, and the method comprises the steps: obtaining a first space relation between a remote controller and a positioner, and obtaining a remote control-positioning space conversion relation between a remote controller coordinate system and a positioner coordinate system; obtaining a second spatial relationship of the projector relative to the positioner, and obtaining a projection-positioning spatial conversion relationship between the projector coordinate system and the positioner coordinate system; obtaining a projection distance and a light path angle, and determining a projection area of a projection picture on the target projection surface in a projector coordinate system; according to the remote control pointing and remote control-positioning space conversion relation in the remote controller coordinate system and the projection area and projection-positioning space conversion relation, intersection point coordinates of the remote control pointing and the projection area are obtained; and displaying an indication cursor at a position corresponding to the intersection point coordinate on the projection picture. The directional remote control of the remote controller on the projection picture is realized, so that the convenience of interactive operation is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of projection equipment, and in particular to an interactive method for projecting images, a projector and a storage medium. Background Art

[0002] Users often have to use remote control buttons to interact with the projected image. When moving a selection box in the projected image, or moving an audio or video progress bar, volume bar, brightness bar, or other directional interactive operations, it is generally necessary to press the corresponding direction buttons on the remote control multiple times to select the corresponding control in the image so that the selection box can reach the corresponding position, which makes the interactive operation cumbersome. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes an interactive method for projecting a picture, a projector and a storage medium, which realize the pointing remote control of the projected picture by a remote controller, thereby improving the convenience of interactive operation.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present invention discloses an interactive method for projecting a picture, comprising the following steps:

[0006] Acquire a first spatial relationship between the remote control and the locator, and obtain a remote control-locating space conversion relationship between a remote control coordinate system of the remote control and a locator coordinate system of the locator according to the first spatial relationship;

[0007] Acquire a second spatial relationship of the projector relative to the locator, and obtain a projection-positioning space conversion relationship between a projector coordinate system of the projector and a locator coordinate system according to the second spatial relationship;

[0008] Obtaining a projection distance of the projector relative to a target projection surface and an optical path angle when the projector projects onto the target projection surface, and determining a projection area of ​​the projection image on the target projection surface in the projector coordinate system according to the projection distance and the optical path angle;

[0009] Obtaining the coordinates of the intersection of the remote control pointing and the projection area according to the conversion relationship between the remote control pointing and the remote control-positioning space in the remote control coordinate system, and the conversion relationship between the projection area and the projection-positioning space;

[0010] A pointer is displayed at a position corresponding to the intersection coordinates on the projection screen.

[0011] Preferably, obtaining a first spatial relationship between the remote controller and the locator comprises:

[0012] Acquire phase differences of multiple groups of first wireless ranging signals between the remote control and the locator, posture parameters of the remote control, and posture parameters of the locator, and determine a first spatial relationship between the remote control and the locator according to the phase differences of the multiple groups of first wireless ranging signals, the posture parameters of the remote control, and the posture parameters of the locator;

[0013] The acquiring of a second spatial relationship of the projector relative to the locator comprises:

[0014] The phase difference of multiple groups of second wireless ranging signals between the projector and the locator, the posture parameters of the projector and the posture parameters of the locator are obtained, and the second spatial relationship between the projector and the locator is determined according to the phase difference of multiple groups of second wireless ranging signals, the posture parameters of the projector and the posture parameters of the locator.

[0015] Preferably, the remote controller is provided with a plurality of first UWB antennas, the projector is provided with a plurality of second UWB antennas, and the locator is provided with a third UWB antenna;

[0016] Respectively acquiring ranging signals from the third UWB antenna through the plurality of the first UWB antennas to obtain a plurality of groups of first wireless ranging signals between the remote controller and the locator;

[0017] The ranging signals from the third UWB antenna are respectively acquired through the plurality of the second UWB antennas to obtain a plurality of groups of second wireless ranging signals between the projector and the locator.

[0018] Preferably, the acquiring the first spatial relationship of the remote control relative to the locator comprises: obtaining the distance between the origin of the remote control coordinate system and the origin of the locator coordinate system, a first vector in the remote control coordinates, and a second vector in the locator coordinate system according to the phase difference of multiple groups of the first wireless ranging signals, and determining the first gravity vector in the remote control coordinate system according to the posture parameters of the remote control, and determining the third gravity vector in the locator coordinate system according to the posture parameters of the locator;

[0019] The method of obtaining the second spatial relationship of the projector relative to the locator includes: obtaining the distance between the origin of the projector coordinate system and the origin of the locator coordinate system, the third vector in the projector coordinate system and the fourth vector in the locator coordinate system according to the phase difference of multiple groups of the second wireless ranging signals, and determining the second gravity vector in the projector coordinate system according to the posture parameters of the projector, and determining the third gravity vector in the locator coordinate system according to the posture parameters of the locator.

[0020] Preferably, obtaining the remote control-positioning space conversion relationship between the remote control coordinate system of the remote control and the locator coordinate system of the locator according to the first spatial relationship comprises:

[0021] Obtaining a first rotation matrix of the remote control according to an angle between a first gravity vector in the remote control coordinate system and a coordinate axis and a coordinate plane of the remote control coordinate system, wherein the first rotation matrix of the remote control includes a rotation relationship that makes the z-axis in the remote control coordinate system parallel to the first gravity vector;

[0022] Obtaining a first rotation matrix of the locator according to the angle between the third gravity vector in the locator coordinate system and the coordinate axis and the coordinate plane of the locator coordinate system, wherein the first rotation matrix of the locator includes a rotation relationship that makes the z-axis in the locator coordinate system parallel to the third gravity vector;

[0023] According to the first vector and the first rotation matrix of the remote control, a first rotation vector is obtained, and according to the angle between the first rotation vector and the coordinate axis of the remote control coordinate system rotated based on the first rotation matrix of the remote control, a second rotation matrix of the remote control is obtained; according to the second vector and the first rotation matrix of the locator, a second rotation vector is obtained, and according to the angle between the second rotation vector and the coordinate axis of the locator coordinate system rotated based on the first rotation matrix of the locator, a second rotation matrix of the locator is obtained;

[0024] Obtaining a first translation matrix according to the distance between the origin of the remote control coordinate system and the origin of the locator coordinate system, and the angle between the second vector and each coordinate axis of the remote control coordinate system;

[0025] According to the first rotation matrix of the remote control, the second rotation matrix of the remote control, the first rotation matrix of the locator, the second rotation matrix of the locator and the first translation matrix, a remote control-positioning space conversion relationship between the remote control coordinate system and the locator coordinate system is obtained.

[0026] Preferably, obtaining the projection-positioning space conversion relationship between the projector coordinate system of the projector and the locator coordinate system according to the second spatial relationship comprises:

[0027] Obtaining a first rotation matrix of the projector according to an angle between a second gravity vector in the projector coordinate system and a coordinate axis and a coordinate plane of the projector coordinate system, wherein the first rotation matrix of the projector includes a rotation relationship that makes the z-axis in the projector coordinate system parallel to the second gravity vector;

[0028] Obtaining a first rotation matrix of the locator according to the angle between the third gravity vector in the locator coordinate system and the coordinate axis and the coordinate plane of the locator coordinate system, wherein the first rotation matrix of the locator includes a rotation relationship that makes the z-axis in the locator coordinate system parallel to the third gravity vector;

[0029] According to the third vector and the first rotation matrix of the projector, a third rotation vector is obtained, and according to the angle between the third rotation vector and the coordinate axis of the projector coordinate system rotated based on the first rotation matrix of the projector, a second rotation matrix of the projector is obtained; according to the fourth vector and the first rotation matrix of the locator, a fourth rotation vector is obtained, and according to the angle between the fourth rotation vector and the coordinate axis of the locator coordinate system rotated based on the first rotation matrix of the locator, a third rotation matrix of the locator is obtained;

[0030] A second translation matrix is ​​obtained according to the distance between the origin of the projector coordinate system and the origin of the locator coordinate system, and the angle between the fourth vector and each coordinate axis of the projector coordinate system.

[0031] According to the first rotation matrix of the projector, the second rotation matrix of the projector, the first rotation matrix of the locator, the third rotation matrix of the locator and the second translation matrix, a projection-positioning space conversion relationship between the projector and the locator is obtained.

[0032] Preferably, the acquiring of the projection distance of the projector relative to the target projection surface and the optical path angle when the projector projects onto the target projection surface, and determining the projection area of ​​the projection picture on the target projection surface in the projector coordinate system according to the projection distance and the optical path angle comprises:

[0033] Obtaining the inclination angle of the projector relative to the target projection surface and the preset optical-mechanical field angle, and obtaining the optical path angle when the projector projects onto the target projection surface according to the optical-mechanical field angle and the inclination angle;

[0034] The projection distance of the projector relative to the target projection surface is obtained, and the projection area of ​​the projection picture on the target projection surface in the projector coordinate system is determined according to the optical path angle and the projection distance.

[0035] Preferably, obtaining the coordinates of the intersection of the remote control pointing and the projection area according to the conversion relationship between the remote control pointing and the remote control-positioning space in the remote control coordinate system, and the conversion relationship between the projection area and the projection-positioning space, comprises:

[0036] According to the projection area of ​​the projection picture on the target projection surface in the projector coordinate system and the projection-positioning space conversion relationship, establish the plane equation corresponding to the projection picture in the locator coordinate system; according to the operation of the designated straight line of the remote control coordinate system and the remote control-positioning space conversion relationship, obtain the straight line equation mapped by the designated straight line in the locator coordinate system, and the straight line equation represents the coordinates of the remote control in the locator coordinate system and the remote control pointing; according to the plane equation and the straight line equation, calculate the intersection coordinates between the remote control pointing and the projection picture in the locator coordinate system; according to the intersection coordinates between the remote control pointing and the projection picture in the locator coordinate system and the projection-positioning space conversion relationship, obtain the intersection coordinates between the remote control pointing and the projection picture in the projector coordinate system; or,

[0037] According to the projection area of ​​the projection picture on the target projection surface in the projector coordinate system, the plane equation of the projection picture in the projector coordinate system is established; according to the calculation of the designated straight line of the remote control coordinate system and the remote control-positioning space conversion relationship and the projection-positioning space conversion relationship, the straight line equation of the designated straight line mapped in the projector coordinate system is obtained, and the straight line equation represents the coordinates of the remote control in the projector coordinate system and the remote control pointing; according to the plane equation and the coordinates and pointing of the remote control in the projector coordinate system, the coordinates of the intersection between the remote control pointing and the projection picture in the projector coordinate system are calculated.

[0038] In a second aspect, the present invention discloses a projector, comprising a processor and a memory, wherein the memory stores a computer program, and the computer program can be read and executed by the processor to implement the interactive method for projecting a picture as described in the first aspect.

[0039] In a third aspect, the present invention discloses a storage medium storing a computer program, wherein the computer program is configured to be executable by a processor to execute the interactive method for projecting a picture in the first aspect.

[0040] Compared with the prior art, the beneficial effects of the present invention are: the interactive method, projector and storage medium for projecting images proposed in the present invention, wherein the projection area of ​​the projected image in the projector coordinate system and the remote control pointing in the remote control coordinate system are unified into the same coordinate system by acquiring the remote control-positioning space conversion relationship between the remote control coordinate system and the locator coordinate system and the projection-positioning space conversion relationship between the projector coordinate system and the locator coordinate system, and the intersection coordinates of the remote control pointing and the projection area are obtained, and the cursor is further displayed at the intersection coordinates, and corresponding touch operations can be performed accordingly, thereby realizing the remote control of the pointing of the projection image by the remote control, and using the remote control on the projector to realize the touch interaction method of absolute coordinate positioning, thereby improving the convenience of interactive operations.

[0041] In a further solution, the remote control, projector, and locator are each built with a UWB antenna, and the distance and angle between the three in space are measured through wireless signals to realize the perception of the specific direction and position of the remote control pointing to the projector screen, so as to be able to calculate the intersection coordinates between the remote control pointing and the projected screen in the projector coordinate system, and display the indicator cursor at the position of the remote control pointing in the projected screen, thereby realizing wireless air interaction. In addition, the present invention has a built-in UWB antenna in each of the remote control, projector, and locator, which can improve the measurement accuracy of the remote control position and angle, thereby improving the accuracy of the display position of the indicator light spot in the projected screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a usage scenario diagram of the remote control, projector, and locator;

[0043] Figure 2 is a flow chart of an interactive method for projecting a picture according to a first embodiment of the present invention;

[0044] Figure 3 is a specific example of the antenna distribution in the remote control;

[0045] Figure 4 is a specific example of the antenna distribution within the projector;

[0046] Figure 5 is the antenna distribution within a locator of a specific instance;

[0047] Figure 6 It is a relationship diagram between the remote control coordinate system, the projector coordinate system, the locator coordinate system and the target projection surface;

[0048] Figure 7 It is the spatial position relationship diagram between the remote control coordinate system and the locator coordinate system;

[0049] Figure 8 It is the spatial position relationship diagram between the projector coordinate system and the locator coordinate system;

[0050] Fig. 9 is the relationship between the gravity component and each coordinate axis in the projector coordinate system;

[0051] Fig.10 It is a schematic diagram of the relationship between the projector coordinate system and the locator coordinate system after relative rotation;

[0052] Fig.11 It is a schematic diagram for determining the coordinates of the projection area of ​​the projection picture in the projector coordinate system;

[0053] Fig.12 It is a schematic diagram of projecting the x and z axes of the projector coordinate system into the plane of the projection screen;

[0054] Fig.13 It is a schematic diagram of determining the coordinates of the intersection between the remote control pointing and the projected image through the conversion relationship among the remote control coordinate system, the projector coordinate system and the locator coordinate system. DETAILED DESCRIPTION

[0055] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope and application of the present invention.

[0056] It should be noted that when a component is referred to as being "fixed to" or "disposed on" another component, it can be directly on the other component or indirectly on the other component. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, connection can be used for both fixing and circuit / signal communication.

[0057] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0058] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0059] The preferred embodiment of the present invention discloses an interactive method for projecting a screen, by setting a wireless space positioning module between a remote control, a projector and a locator, and based on the space (distance and direction) information obtained by the wireless space positioning module on the remote control, the projector and the locator, the pointing position of the remote control on the projected screen is determined. The locator is a separate accessory independent of the projector, placed in front of the projector, but does not block the projected screen, and is usually placed outside the projection area, such as below the projected screen.

[0060] Among them, the technical types of wireless space positioning may be, for example, UWB (Ultra Wide Band) technology, Bluetooth BLE (Bluetooth Low Energy) technology, star flash, or microwave radar signals with custom protocols. These wireless space positioning technologies all have the functions of measuring distance and the angle of arrival of electromagnetic wave signals.

[0061] The following uses UWB technology to illustrate how to calculate the distance and azimuth angle between the remote control and the projector. For example, the TOA (Time of arrival) ranging method, the AOA (Angle of Arrival) positioning method or the PDOA (Phase Difference of Arrival) angle measurement method can be used.

[0062] Among them, the TOA ranging method measures the propagation time from the transmitter to the receiver and multiplies it by the speed of the electromagnetic wave to get the distance. This method requires strict clock synchronization between the transmitter and the receiver, otherwise there will be time deviation, resulting in ranging deviation. The TW-TOF (Two Way Time of Flight) method based on the TOA ranging method is more commonly used. For example, to measure the distance between A and B, first A is used as the transmitter and B is used as the receiver - A sends and B receives, then B is used as the transmitter and A is used as the receiver - B sends and A receives, where the time difference between A sending and receiving is ta, and the time difference between B sending and receiving is tb. The time difference between ta and tb is the time it takes for the electromagnetic wave to go back and forth between A and B. Based on ta and tb, the distance between A and B can be calculated. This method does not require A and B clock synchronization, but in order to calculate the accuracy of the absolute value of time, high clock accuracy is required, otherwise it will cause deviation. In order to reduce the ranging error caused by clock deviation, measurements in both forward and reverse directions can be used. For example, first measure in the forward direction and then in the reverse direction. That is, B sends and A receives, then A responds and sends and B receives. By measuring in the forward and reverse directions multiple times and averaging them, the impact of clock deviation can be reduced and the accuracy of ranging can be improved.

[0063] The AOA positioning method can measure the incident angle of electromagnetic waves from an unknown point to a reference point, which is usually obtained by the receiver detecting the source direction of the signal energy peak through the antenna array, or by the antenna array measuring the phase difference of the signal.

[0064] The PDOA angle measurement method is to set two antenna receiving ends on the receiving device and one transmitting antenna on the transmitting device in a two-dimensional space, where the distance d between the two antenna receiving ends is known. The time difference of the received signals is calculated based on the phase difference of the received signals at the two antenna receiving ends, and the arrival angle of the transmitting end can be calculated. The specific formula is as follows:

[0065] p=d·sinθ p

[0066]

[0067] When two receiving antennas are in the remote control, the transmitting antenna can be set in the projector; when two receiving antennas are in the projector, the transmitting antenna is set in the remote control. The transmitting antenna transmits a wireless signal with a wavelength of λ. p is the distance difference between the transmitting antenna and the two receiving antennas, d is the distance between the two receiving antennas, and d is a known value; α p It represents the arrival phase difference of the wireless signal from the transmitting antenna received by the two receiving antennas, and λ is the wavelength of the wireless signal from the transmitting antenna. p It is the angle between the transmitting antenna and the line connecting the two receiving antennas, and is used to characterize the angle between the projector and the axis of the remote control.

[0068] Among them, the PDOA angle measurement method requires that the two receiving antennas are very close to each other; further, the optimal spacing between the two receiving antennas is half the wavelength, that is, λ / 2. The optimal antenna spacing refers to the spacing that maximizes the arrival phase difference without generating phase ambiguity. The basic principle of the PDOA angle measurement method is to convert the arrival angle θ p Converted to arrival phase difference α p , the arrival angle is estimated indirectly by measuring the arrival phase difference. p Convert to alpha p The larger the magnification of , the higher the estimation accuracy. When the antenna spacing is the optimal spacing, θ p ∈[-π / 2,π / 2], θ p With α p There is a one-to-one correspondence.

[0069] In summary, the distance between the transmitting antenna and the receiving antenna array of each wireless spatial positioning module in three-dimensional space and the angle of arrival between them can be calculated through TOA distance measurement and PDOA angle measurement. In addition to the methods given here, there are other methods to measure the pointing point of the remote control on the projection screen, which are not listed here one by one.

[0070] like Figure 1 As shown, it is a usage scenario diagram of the remote control 10, the projector 20 and the locator 30, wherein the optical center of the projector 20 is marked as O2, and the optical center O2 can be understood as the convergence point of the projection light path in the optical machine. At the same time, since the distance between the wireless space positioning module in the projector 20 and the optical machine is much smaller than the distance from the projector 20 to the locator 30, the coordinates of the antenna origin in the wireless space positioning module can be approximately regarded as the optical center O2 of the optical machine; the four vertices of the projection surface projected by the projector 20 on the target projection surface are P1, P2, P3, and P4, and the target projection surface is generally a wall or a curtain; the origin of the antenna in the remote control 10 is O1, and the intersection of the remote control 20 pointing to the target projection surface is P'; the antenna origin of the locator 30 is marked as O3. Among them, the spatial relationship between the antenna origin O3 of the locator 30 and the optomechanical optical center O2 of the projector 20 is used to automatically calibrate the spatial geometric relationship between the optomechanical projection image of the projector 20 and the locator 30; the spatial relationship between the antenna origin O1 of the remote control 10 and the antenna origin O3 of the locator 30 is used to calculate the spatial geometric relationship between the remote control pointing and the locator 30. To realize visual control of the projection image through remote control pointing, it is necessary to display an indicator cursor at the position of the projection image in the target projection surface pointed by the remote control, so it is necessary to calculate the absolute coordinates of the intersection of the remote control pointing and the projection image, which is converted into a mathematical problem, that is, it is necessary to find the three-dimensional coordinates of the intersection point P' of the straight line O1P' and the target projection surface in the projector coordinate system O2. According to the angle measurement and distance measurement principles of the aforementioned TOA ranging method and PDOA angle measurement method, the antenna layout of the remote control 10, projector 20 and the locator 30 is set respectively, as well as the mathematical relationship between the antenna and the three-dimensional coordinate system.

[0071] like Figure 2 As shown, the interactive method for projecting a picture according to the first embodiment of the present invention comprises the following steps:

[0072] S1: obtaining a first spatial relationship of the remote control relative to the locator, and obtaining a remote control-locating space conversion relationship between a remote control coordinate system of the remote control and a locator coordinate system of the locator according to the first spatial relationship; obtaining a second spatial relationship of the projector relative to the locator, and obtaining a projection-locating space conversion relationship between a projector coordinate system of the projector and a locator coordinate system according to the second spatial relationship;

[0073] In the example of UWB technology, a plurality of first UWB antennas are provided in the remote control, a plurality of second UWB antennas are provided in the projector, and a third UWB antenna is provided in the locator.

[0074] like Figure 3As shown, it is an antenna distribution in a remote control in a specific example, through which the signal incident angle can be measured in the remote control coordinate system, wherein antennas A1, B1, and C1 are respectively antennas of the wireless space positioning module in the remote control 10, which can be regarded as the first UWB antenna, and the position of antenna B1 is the origin O1 of the remote control coordinate system, that is, the origin corresponding to the antenna in the above remote control 10, the line connecting antenna A1 and antenna B1 is perpendicular to the horizontal distribution of the remote control and parallel to the height direction of the remote control 10; the line connecting antenna B1 and antenna C1 is perpendicular to the remote control pointing distribution and parallel to the width direction of the remote control 10; antenna D1 is the antenna of the transmitting end, for example, the transmitting antenna of the locator 30.

[0075] like Figure 4 As shown, it is an antenna distribution in a projector 20 in a specific example, through which the signal incident angle can be measured in the projector coordinate system, wherein antennas A2, B2, and C2 are antennas of the wireless spatial positioning module in the projector 20, respectively, and can be regarded as the second UWB antenna, the position of antenna B2 is the origin O2 of the projector coordinate system, the line connecting antenna A2 and antenna B2 is perpendicular to the horizontal plane of the projector body, and parallel to the height direction of the projector 20; the line connecting antenna B2 and antenna C2 is perpendicular to the optical axis direction (the direction of the central projection light path), and parallel to the width direction of the projector 20; antenna D2 is the antenna of the transmitting end, for example, the transmitting antenna of the locator 30. Among them, the three antennas set in the projector 20 only need to ensure that the spatial fixed position relationship of antennas A2, B2, and C2 is associated with the up, down, front, back, left, and right directions of the projector. For example, antenna A2 can also be arranged at intervals in the direction of the optical axis of antenna B2, or antenna C2 can be arranged at intervals in the direction of the optical axis of B2. Generally, as long as A2B2 is perpendicular to B2C2, and A2B2 and B2C2 are respectively parallel to any different axis of the three axes of the projector (left-right axis, front-back axis, and up-down axis), it can be used. Because the position of each antenna in the projector coordinate system can be calculated by the above-mentioned TOA ranging method and PDOA angle measurement method, when the physical space arrangement of these antennas in the projector satisfies: A2B2 is parallel to the height direction of the projector, and B2C2 is parallel to the width direction of the projector, the coordinate position relationship of each antenna in the space coordinate system established in the software program can be calculated by the TOA ranging method and the PDOA angle measurement method, and then the direction of antenna A2B2 in the projector coordinate system can be understood as the up-down direction of the projector, the direction of B2C2 is the left-right direction, and the direction passing through B2 and perpendicular to the A2B2C2 plane is the optical axis direction.

[0076] like Figure 5As shown, it is a specific example of the antenna distribution in the locator, wherein antennas A3, B3, and C3 are respectively the antennas of the wireless space positioning module in the locator, which can be regarded as the third UWB antenna. The position of antenna B3 corresponds to the origin O3 of the locator coordinate system. The line connecting antenna A3 and antenna B3 is perpendicular to the horizontal plane of the locator. The y-axis of the locator coordinate system is perpendicular to the side of the locator facing the projector 20 and the remote control 10. The line connecting antenna B3 and antenna C3 is perpendicular to the y-axis. Antenna D3 is the antenna of the transmitting end, for example, the transmitting antenna in the wireless space positioning module of the projector 20 or the remote control 10.

[0077] In one embodiment, step S1 specifically includes:

[0078] S11: Acquire a first spatial relationship between the remote controller and the locator;

[0079] Specifically, the phase difference of multiple groups of first wireless ranging signals between the remote control 10 and the locator 30, the posture parameters of the remote control and the posture parameters of the locator are obtained, and the first spatial relationship between the remote control and the locator is determined according to the phase difference of multiple groups of first wireless ranging signals, the posture parameters of the remote control and the posture parameters of the locator. Among them, the ranging signals from the third UWB antenna in the controller 30 are respectively obtained through the multiple first UWB antennas in the remote control 10 to obtain multiple groups of first wireless ranging signals between the remote control 10 and the locator 30, and the first UWB antenna is, for example, the aforementioned antenna A1, A2, A3, and the third UWB antenna is, for example, the aforementioned antenna D1. Through TOA ranging method, PDOA angle measurement method and other measurement methods, the spatial relationship of antennas A1, A2, A3 relative to antenna D1 can be calculated, and then the direction and distance of the remote control 10 relative to the locator 30 can be obtained.

[0080] Furthermore, the distance between the origin O1 of the remote control coordinate system and the origin O3 of the locator coordinate system, the first vector in the remote control coordinates and the second vector in the locator coordinate system are obtained according to the phase difference of multiple groups of first wireless ranging signals, and the first gravity vector in the remote control coordinate system is determined according to the posture parameters of the remote control, and the third gravity vector in the locator coordinate system is determined according to the posture parameters of the locator.

[0081] like Figure 3In a specific example, the beam receiving directions of antennas A1, B1, and C1 of the remote controller 10 are parallel to the remote control direction (i.e., the y-axis of the remote control coordinate system) and face the transmitting antenna D1 of the locator. This layout can maximize the effective angle measurement range of the remote control receiving antenna array (including antennas A1, B1, and C1). The optimal spacing between antennas is λ / 2, and the antenna spacing is in the centimeter scale. When calculating the distance and angle between the transmitting source and the receiving antenna, the antenna spacing can be approximately regarded as 0. Therefore, the distances between antennas A1, B1, and C1 and antenna D1 can be regarded as the distance between the origin O1 (antenna B1) and antenna D1. The angle between the line connecting antennas A1 and B1 and the line connecting antennas D1 and B1, as well as the angle between the line connecting antennas B1 and C1 and the line connecting antennas D1 and B1, all belong to the angle between the coordinate axis passing through the origin O1 (antenna B1) and antenna D1 to the origin O1. The distance between antenna D1 and antenna B1 is obtained by TOA ranging method. The angle ∠D1B1C1 between D1B1 and B1C1 is represented by ω1 and can be obtained by PDOA angle measurement method. The angle ∠A1B1D1 between D1B1 and A1B1 is obtained by PDOA angle measurement method. The projection of antenna D1 on the xy plane is D1ˊ, and the angle between D1B1 and D1ˊB1 is θ1=90°-∠A1B1D1. According to the spatial three cosine theorem, cosω1=cosθ1*cosΦ1, the angle Φ1 between D1ˊB1 and B1C1 can be obtained. Therefore, in the projector coordinate system, the y-axis is parallel to the optical axis, the xy plane is parallel to the horizontal plane of the projector body, and the z-axis is perpendicular to the xy plane. The angle between the line connecting the antenna D1 at the transmitting end and the origin O1 relative to the x, y, and z axes, as well as the angle between the projection of the antenna D1 on the xy / yz / xz plane and the x / y / z axes can be calculated by the PDOA angle measurement method. Based on the distance and angle information calculated above, a remote control coordinate system can be established, and the coordinates of the antenna D1 can be obtained in the remote control coordinate system. Similarly, a locator coordinate system and a projector coordinate system are also established according to the above measurement method. The coordinates of the antenna B1 in the remote control 10 and the B2 in the projector 20 can be obtained in the locator coordinate system, and the coordinates of the antenna B3 in the locator 30 can be obtained in the projector coordinate system.

[0082] In order to simplify the calculation, in this specific example, the center point of the antenna approximating the projector 20 coincides with the optical center, and the result is as follows: Figure 6 The schematic diagram of the antenna layout and spatial relationship of the remote control 10, projector 20, and locator 30 is shown. Among them, the locator 30 is in front of the projection screen, facing the remote control 10 and the projector 20, and the antenna beam angle of the locator 30 only needs to meet the general 120° to cover the pointing range of the remote control 10. In this specific example, the remote control 10, the projector 20, and the locator 30 only need to have a set of 1T3R wireless modules built in each to realize the positioning of the intersection point P' of the remote control pointing and the projection screen in the three-dimensional space in the projector coordinate system.

[0083] The locator 30 can be placed in front of the projector 20 and the remote control 10 with the antenna side of the locator 30 facing the remote control 10 and the projector 20. It does not need to be placed in a fixed position relative to the projection screen, and there is no need to manually calibrate the spatial relationship between the projection screen and the locator 30 during use.

[0084] The remote controller 10 , the projector 20 , and the positioner 30 each have a built-in IMU (Inertial Measurement Unit) module for measuring the gravity acceleration components of the three spatial axes, and for calculating the pitch angle and roll angle thereof relative to the horizontal calibration plane.

[0085] In this specific example, the remote control coordinate system O1 is the spatial three-dimensional coordinate system of the remote control antenna, the y-axis in this coordinate system coincides with the pointing direction of the remote control, and the O1xy plane is the horizontal plane of the remote control; the projector coordinate system O2 is the spatial three-dimensional coordinate system of the projector antenna, the y-axis in this coordinate system coincides with the optical axis of the projector optical machine, and the O2xy plane is parallel to the horizontal plane of the projector body; the locator coordinate system O3 is the spatial three-dimensional coordinate system of the locator antenna, the y-axis in this coordinate system is perpendicular to the side of the locator facing the projector and the remote control, and the O3xy plane is the horizontal plane of the locator.

[0086] In this step, the TOA distance measurement method is used to measure the straight-line distance O1O3 between the origins of the remote control coordinate system O1 and the locator coordinate system O3; the PDOA angle measurement method is used to measure the vector The spatial angle between the coordinate axes in each coordinate system; based on this, the equation of the straight line pointing to the locator coordinate system O3 can be calculated later.

[0087] Among them, Figure 7 As shown, the first vector in the remote control coordinate system O1 is The second vector in the locator coordinate system O3 is Wherein, θ, Φ, ω are the three-axis angles between the straight line O1O3 and the locator coordinate system O3: the angle between O1O3 and the x-axis is θ, the angle between O1O3 and the y-axis is Φ, and the angle between O1O3 and the z-axis is ω; θ′, Φ′, ω′ are the three-axis angles between the straight line O1O3 and the remote control coordinate system O1: the angle between O1O3 and the x-axis is θ′, the angle between O1O3 and the y-axis is Φ′, and the angle between O1O3 and the z-axis is ω′. According to the principle of PDOA angle measurement method, θ, Φ, ω and θ′, Φ′, ω′ can be measured.

[0088] The gravity g vector is used to match the remote control coordinate system O1 and the locator coordinate system O3 to the same vertical direction. The remote control and the locator have their own IMU modules (inertial measurement modules) that can measure gravity acceleration to calibrate the z-axis direction of the remote control coordinate system and the locator coordinate system. In one embodiment, the remote control and the locator are both equipped with IMU modules, and the first gravity vector and the third gravity vector can be determined by the following method:

[0089] The first attitude angle data of the IMU module in the remote controller 10 is obtained, and the second gravity vector is determined in the remote controller coordinate system according to the first attitude angle data; the third attitude angle data of the IMU module in the locator 30 is obtained, and the third gravity vector is determined in the locator coordinate system according to the third attitude angle data. The x-axis or y-axis of the remote controller coordinate system O1 is toward the front of the remote controller, that is, the direction of the side where the remote controller is pointing; the z-axis is toward the top of the remote controller, so the first attitude angle data measured by the IMU module in the remote controller 10 can directly reflect the direction of the gravity of the remote controller in the remote controller coordinate system O1, so that the first gravity vector can be established, and the vector can be a unit vector, and the first attitude angle data includes the yaw angle, roll angle, and pitch angle information of the remote controller. Similarly, the third attitude angle data measured by the IMU module in the locator 30 can directly reflect the direction of the gravity of the locator in the locator coordinate system O3, so that the third gravity vector can be established, and the vector can be a unit vector, and the third attitude angle data includes the yaw angle, roll angle, and pitch angle information of the locator.

[0090] S12: Obtaining a remote control-positioning space conversion relationship between a remote control coordinate system of the remote control and a locator coordinate system of the locator according to the first spatial relationship;

[0091] Specifically, step S12 includes:

[0092] S121: obtaining a first rotation matrix of the remote control according to an angle between a first gravity vector in the remote control coordinate system and a coordinate axis and a coordinate plane of the remote control coordinate system, wherein the first rotation matrix of the remote control includes a rotation relationship that makes a z-axis in the remote control coordinate system parallel to the first gravity vector;

[0093] S122: According to the angle between the third gravity vector in the locator coordinate system and the coordinate axis and coordinate plane of the locator coordinate system, the first rotation matrix of the locator is obtained, and the first rotation matrix of the locator includes a rotation relationship that makes the z-axis in the locator coordinate system parallel to the third gravity vector; the first rotation matrix of the remote control and the first rotation matrix of the locator are used to rotate the z-axis of the remote control coordinate system and the locator coordinate system to be parallel to the direction of gravity.

[0094] S123: According to the first vector and the first rotation matrix of the remote controller, a first rotation vector is obtained, and according to the angle between the first rotation vector and the coordinate axis of the remote controller coordinate system rotated based on the first rotation matrix of the remote controller, a second rotation matrix of the remote controller is obtained; according to the second vector and the first rotation matrix of the locator, a second rotation vector is obtained, and according to the angle between the second rotation vector and the coordinate axis of the locator coordinate system rotated based on the first rotation matrix of the locator, a second rotation matrix of the locator is obtained; the second rotation matrix of the remote controller and the second rotation matrix of the locator are used to rotate the x-axis of the remote controller coordinate system and the locator coordinate system to be parallel. Through the action of the first rotation matrix of the remote controller, the second rotation matrix of the remote controller, the first rotation matrix of the locator, and the second rotation matrix of the locator, the remote controller coordinate system and the locator coordinate system can be rotated to a state where the corresponding axes are parallel, thereby realizing the coordinate conversion in the coordinate system. The first vector is the vector of the line connecting the origin of the remote controller coordinate system O1 and the origin of the locator coordinate system O3 in the remote controller coordinate system O1, and the first rotation vector is the vector of the line connecting the remote controller coordinate system O1 after the remote controller coordinate system O1 is rotated by the first rotation matrix of the remote controller in the rotated remote controller coordinate system O1. The second vector is the vector of the line connecting the origin of the remote control coordinate system O1 and the origin of the locator coordinate system O3 in the locator coordinate system O3. The second rotation vector is the vector of the line connecting the origin of the remote control coordinate system O1 and the origin of the locator coordinate system O3 in the locator coordinate system O3 after the locator coordinate system is rotated by the first rotation matrix of the locator.

[0095] S124: Obtain a first translation matrix according to the distance between the origin of the remote control coordinate system and the origin of the locator coordinate system, and the angle between the second vector and each coordinate axis of the remote control coordinate system;

[0096] S125: Obtain a remote control-positioning space conversion relationship between the remote control coordinate system and the positioner coordinate system according to the remote control first rotation matrix, the remote control second rotation matrix, the positioner first rotation matrix, the positioner second rotation matrix and the first translation matrix.

[0097] In a specific example, Figure 7 As shown, the coordinates of the origin of the remote control coordinate system O1 relative to the origin of the positioner coordinate system O3 are located at L1{cosθ, cosΦ, cosω}, which corresponds to the second vector It can be obtained that the matrix transformation from the remote control coordinate system O1 to the locator coordinate system O3 can be expressed by the following rotation + translation:

[0098]

[0099] Wherein, L1 is the distance between the remote control antenna group and the locator antenna group, that is, the distance O1O3 between the origin of the remote control coordinate system O1 and the origin of the locator coordinate system O3, R1 is the rotation matrix from the remote control coordinate system O1 to the locator coordinate system O3, R1 is composed of the first rotation matrix of the remote control, the second rotation matrix of the remote control, the first rotation matrix of the locator, and the second rotation matrix of the locator. is the first translation matrix.

[0100] Define the above coordinate rotation + translation conversion matrix as T1. So far, the remote control-positioning space conversion relationship between the remote control 10 and the locator 30 can be obtained according to the above formula, wherein the specific derivation process of the R1 rotation matrix and the translation matrix is ​​the same as the derivation step of the conversion matrix T2 in the subsequent step S14, and will not be repeated here. (x', y', z') is the coordinate of any point in the remote control coordinate system O1, and (x, y, z) is the corresponding coordinate of (x', y', z') in the locator coordinate system O3 after being converted by the conversion matrix T1.

[0101] S13: Acquire a second spatial relationship between the projector and the locator;

[0102] Specifically, the phase difference of multiple sets of second wireless ranging signals between the projector and the locator, the posture parameters of the projector and the posture parameters of the locator are obtained, and the second spatial relationship between the projector and the locator is determined according to the phase difference of multiple sets of second wireless ranging signals, the posture parameters of the projector and the posture parameters of the locator. The ranging signals from the third UWB antenna are respectively obtained through multiple second UWB antennas to obtain multiple sets of second wireless ranging signals between the projector and the locator.

[0103] Furthermore, the distance between the origin of the projector coordinate system and the origin of the locator coordinate system, the third vector in the projector coordinate system and the fourth vector in the locator coordinate system are obtained according to the phase difference of multiple groups of second wireless ranging signals, and the second gravity vector in the projector coordinate system is determined according to the posture parameters of the projector, and the third gravity vector in the locator coordinate system is determined according to the posture parameters of the locator.

[0104] like Figure 4In a specific example, the beam directions of antennas A2, B2, and C2 in projector 20 are parallel to the optical axis of the projector (i.e., the y-axis of the projector coordinate system) and face the transmitting antenna D2 of the locator 30. This layout can maximize the effective angle measurement range of the projector receiving antenna array (including antennas A2, B2, and C2). The optimal spacing between antennas is λ / 2, and the antenna spacing is in the centimeter scale. When calculating the distance and angle between the transmitting source and the receiving antenna, the antenna spacing can be approximately regarded as 0. Therefore, the distances between antennas A2, B2, and C2 and antenna D2 can be regarded as the distances between the origin (antenna B2) and antenna D2, and the angles between the line connecting antennas A2 and B2 and antenna D2, as well as the angles between the line connecting antennas B2 and C2 and antenna D2, can be regarded as the angles between the coordinate axis passing through the origin (antenna B2) and antenna D2. The distance between antenna D2 and antenna B2 is obtained by TOA ranging method. The angle ∠D2B2C2 between D2B2 and B2C2 is represented by α2, which can be obtained by PDOA angle measurement method. The angle ∠A2B2D2 between D2B2 and A2B2 is represented by γ2, which can be obtained by PDOA angle measurement method. The projection of antenna D2 on the xy plane is D2ˊ, and the angle between D2B2 and D2ˊB2 is θ2=90°-γ2. According to the spatial three cosine theorem: cosα2=cosθ2*cosΦ2, the angle Φ2 between D2ˊB2 and B2C2 can be obtained. Similarly, according to the spatial three cosine theorem: cosβ2=cosθ2*cos(90°-Φ2), the angle β2 between D2B2 and the y-axis can be obtained. Therefore, in the projector coordinate system, the y-axis is parallel to the optical axis, the xy plane is parallel to the horizontal plane of the projector body, and the z-axis is perpendicular to the xy plane. The angles between the transmitting antenna D2 and the x, y, and z axes, as well as the angles between the projection of the antenna D2 on the xy / yz / xz plane and the x / y / z axes can be calculated using the PDOA angle measurement method.

[0105] In this step, the straight-line distance O2O3 between the origins of the projector coordinate system O2 and the locator coordinate system O3 is measured by the TOA distance measurement method; the vector is measured by the PDOA angle measurement method. The spatial angle with each receiving antenna; used to calculate the spatial coordinate relationship of the projection image in the locator coordinate system O3.

[0106] Among them, Figure 8 As shown, the third vector in the projector coordinate system O2 is The fourth vector in the locator coordinate system O3 is Wherein, α, β, γ are the three-axis angles between the straight line O2O3 and the locator coordinate system O3: the angle between O2O3 and the x-axis is α, the angle between O2O3 and the y-axis is β, and the angle between O2O3 and the z-axis is γ; α′, β′, γ′ are the three-axis angles between the straight line O2O3 and the projector coordinate system O2: the angle between O2O3 and the x-axis is α′, the angle between O2O3 and the y-axis is β′, and the angle between O2O3 and the z-axis is γ′. According to the principle of PDOA angle measurement method, α, β, γ and α′, β′, γ′ can be measured.

[0107] The projector coordinate system O2 and the locator coordinate system O3 are matched to the same vertical direction by means of the gravity g vector. The projector and the locator have their own IMU modules (inertial measurement modules) that can measure gravity acceleration. In one embodiment, both the projector and the locator are provided with IMU modules, and the second gravity vector and the third gravity vector can be determined by the following method:

[0108] The second attitude angle data of the IMU module in the projector is obtained, and the second gravity vector is determined in the projector coordinate system according to the second attitude angle data; the third attitude angle data of the IMU module in the locator is obtained, and the third gravity vector is determined in the locator coordinate system according to the third attitude angle data. The x-axis or y-axis of the projector coordinate system O2 is toward the front of the projector, that is, the direction of the side where the optical machine light outlet is located; the z-axis is toward the top of the projector, so the second attitude angle data measured by the IMU module in the projector can directly reflect the direction of the projector gravity in the projector coordinate system O2, so that the second gravity vector can be established, and the vector can be a unit vector, and the second attitude angle data includes the yaw angle, roll angle, and pitch angle information of the projector. Similarly, the third attitude angle data measured by the IMU module in the locator can directly reflect the direction of the gravity of the locator in the locator coordinate system, so that the third gravity vector can be established, and the vector can be a unit vector, and the third attitude angle data includes the yaw angle, roll angle, and pitch angle information of the locator.

[0109] S14: Obtaining a projection-positioning space conversion relationship between a projector coordinate system of the projector and a positioner coordinate system according to the second spatial relationship;

[0110] In a specific example, by obtaining the rotation matrix and translation matrix from the projector coordinate system O2 to the locator coordinate system O3, the projection-positioning space conversion relationship between the projector coordinate system O2 and the locator coordinate system O3 can be obtained, which specifically includes the following steps:

[0111] S141: obtaining a first rotation matrix of the projector according to an angle between a second gravity vector in the projector coordinate system and a coordinate axis and a coordinate plane of the projector coordinate system, wherein the first rotation matrix of the projector includes a rotation relationship that makes the z-axis in the projector coordinate system parallel to the second gravity vector;

[0112] like Fig. 9 In the projector coordinate system O2, the components of gravity g on the x, y, and z axes, and the projection component of g on the yz plane g' are known. yz , we can get g' yz Angle with z axis Ф' yz , g and g' yz The angle θ' yz The projector coordinate system O2 rotates around the x-axis by Ф' yz , get the rotation matrix R' x (Ф' yz ), and then rotate around the y axis by θ' yz , get the rotation matrix R' y (θ' yz ); After two rotations, the z axis of the coordinate system O2 coincides with g. In this specific example, g' yz The angle between the coordinate axis z and the projector coordinate system is Ф' yz , the angle between the second gravity vector in the projector coordinate system and the coordinate plane yO2z is θ' yz , the projector first rotation matrix includes R' x (Ф' yz ) and R' y (θ' yz ). After the projector first rotation matrix R' x (Ф' yz ) and R' y (θ' yz ), the z-axis of the projector coordinate system O2 can be rotated to be parallel to the direction of gravity.

[0113] S142: obtaining a first rotation matrix of the locator according to an angle between a third gravity vector in the locator coordinate system and a coordinate axis and a coordinate plane of the locator coordinate system, wherein the first rotation matrix of the locator includes a rotation relationship that makes the z-axis in the locator coordinate system parallel to the third gravity vector;

[0114] Similarly, in the locator coordinate system O3, the components of gravity g on the x, y, and z axes, and the projection component of g on the yz plane are known. yz , we can get g yz Angle Ф with z axis yz , g and g yz The angle θ yz ; The locator coordinate system O3 first rotates around the x-axis by Ф yz , get the rotation matrix R x (Ф yz ), and then rotate θ around the y axis yz , get the rotation matrix R y (θ yz); After two rotations, the z axis of the locator coordinate system O3 coincides with the g axis. In this specific example, the angle between the third gravity vector in the locator coordinate system O3 and the coordinate axis of the locator coordinate system O3 includes Ф yz , the angle between the third gravity vector in the locator coordinate system O3 and the coordinate plane includes θ yz , the first rotation matrix of the locator includes R x (Ф yz ) and R y (θ yz ). After the first rotation matrix R of the locator x (Ф yz ) and R y (θ yz ), the z-axis of the locator coordinate system O3 can be rotated to be parallel to the direction of gravity.

[0115] S143: According to the third vector and the first rotation matrix of the projector, a third rotation vector is obtained, and according to the angle between the third rotation vector and the coordinate axis of the projector coordinate system rotated based on the first rotation matrix of the projector, the second rotation matrix of the projector is obtained; according to the fourth vector and the first rotation matrix of the locator, a fourth rotation vector is obtained, and according to the angle between the fourth rotation vector and the coordinate axis of the locator coordinate system rotated based on the first rotation matrix of the locator, the third rotation matrix of the locator is obtained.

[0116] Among them, the third vector is the vector of the line connecting the origin of the projector coordinate system O2 and the origin of the locator coordinate system O3 in the projector coordinate system O2, and the third rotation vector is the vector of the line in the rotated projector coordinate system O2 after the projector coordinate system O2 is rotated by the projector first rotation matrix. The fourth vector is the vector of the line connecting the origin of the projector coordinate system O2 and the origin of the locator coordinate system O3 in the locator coordinate system, and the fourth rotation vector is the vector of the line in the rotated locator coordinate system O3 after the locator coordinate system O3 is rotated by the locator first rotation matrix.

[0117] like Fig.10 , after the rotation of the projector coordinate system O2 and the locator coordinate system O3, the z axis coincides with g, and the xy planes of the two are parallel. The third vector After the corresponding straight line is transformed by the first rotation matrix of the projector, the angle between it and the xyz axis of the rotated projector coordinate system becomes α' g , β' g , γ' g ; The fourth vector After the corresponding straight line is transformed by the first rotation matrix of the locator, the angles between it and the xyz axis of the rotated locator coordinate system become α g , β g , γ gCalculate the angle θ' between O2O3 and the projection of O2O3 on the xy plane in the rotated projector coordinate system O2 g =|90°-γ' g |, through the three cosine theorem of space angle cosα' g = cosθ' g *cosФ' g The angle Ф between the projection of O2O3 on the xy plane of the projector coordinate system O2 and the x-axis can be obtained g Similarly, the angle Ф between the projection of O2O3 on the xy plane of the locator coordinate system O3 and the x-axis can be obtained: g The projector coordinate system O2 is then rotated around the z axis by Ф' g , get the rotation matrix R' z (Ф' g ), let the x-axis be The projection of the vector on the xy plane is aligned; the locator coordinate system O3 is also rotated around the z axis by Ф g , get the rotation matrix R z (Ф g ), let the x-axis be The projection of the vector in the xy plane is aligned. After these rotation transformations, the xyz axes of the projector coordinate system O2 and the locator coordinate system O3 are aligned parallel in space. In this specific example, the third rotation vector is the new vector of the third vector in the rotated projector coordinate system O2 when the projector coordinate system O2 is rotated so that the z axis coincides with g. The angle between the third rotation vector and the coordinate axis of the rotated projector coordinate system O2 includes α' g , β' g , γ' g The fourth rotation vector is a new vector of the fourth vector in the rotated locator coordinate system O3 when the locator coordinate system O3 rotates so that the z axis coincides with g. The angle between the fourth rotation vector and the coordinate axis of the rotated locator coordinate system O3 includes α g , β g , γ g , the second rotation matrix of the projector includes R' z (Ф' g ), the third rotation matrix of the locator includes R z (Ф g ).

[0118] Correspondingly, the second rotation matrix of the locator is determined by the projection of the line connecting the origin of the remote control coordinate system O1 to the origin of the locator coordinate system O3 on the xy plane, and the angle between the x-axis of the locator coordinate system after rotation by the first rotation matrix of the locator, and the second rotation matrix of the locator rotates the locator coordinate system O3 around the z-axis by the angle. The second rotation matrix of the remote control is determined by the projection of the line connecting the origin of the remote control coordinate system O1 to the origin of the locator coordinate system O3 on the xy plane, and the angle between the x-axis of the remote control coordinate system after rotation by the first rotation matrix of the remote control, and the second rotation matrix of the remote control rotates the remote control coordinate system O1 around the z-axis by the angle. The coordinate axes of the remote control coordinate system O1 after rotation by the first rotation matrix of the remote control and the second rotation matrix of the remote control are respectively parallel to the corresponding coordinate axes of the locator coordinate system O3 after rotation by the first rotation matrix of the locator and the second rotation matrix of the locator.

[0119] S144: Obtain a second translation matrix according to the distance between the origin of the projector coordinate system and the origin of the locator coordinate system, and the angle between the fourth vector and each coordinate axis of the projector coordinate system;

[0120] The coordinates of the origin of O3 in the projector coordinate system O2 have been obtained above. The distance O2O3 between the origin of the projector coordinate system O2 and the origin of the locator coordinate system O3 represents the distance between the projector antenna group and the locator antenna group, which is set to L2 and can be calculated according to the TOA ranging method. The fourth vector of the origin of the projector coordinate system O2 relative to the origin of the locator coordinate system O3 in the locator coordinate system O3 is The second translation matrix of the origin of the projector coordinate system O2 relative to the origin of the locator coordinate system O3 is calculated as: L2 is the distance between the origin of the projector coordinate system O2 and the origin of the locator coordinate system O3, which can be measured by the TOA method.

[0121] S145: Obtain a projection-positioning space conversion relationship between the projector and the positioner according to the projector first rotation matrix, the projector second rotation matrix, the positioner first rotation matrix, the positioner third rotation matrix and the second translation matrix.

[0122] Combining the above rotation transformation process, it can be obtained that after the following rotation transformations, the xyz axes of the projector coordinate system O2 and the positioner coordinate system O3 are parallel and aligned in space: the projector coordinate system O2 is first rotated around the x-axis by Ф' yz , get the rotation matrix R' x (Ф' yz ), and then rotate around the y axis by θ' yz , get the rotation matrix R' y (θ' yz ), at this time, the z axis is aligned with the gravity g vector; then rotate around the z axis Ф'g , get the rotation matrix R' z (Ф' g ), at this time the x-axis and Align the projection of the vector in the xy plane; then rotate the projector coordinate system O2 in the opposite direction around the z axis. g , that is, R z (Ф g ) Then rotate θ in the opposite direction around the y axis yz , that is, R y (θ yz ) Then rotate Φ in the opposite direction around the x-axis yz , that is, R x (Ф yz ) This will make the coordinate axes of the projector coordinate system O2 parallel to the corresponding coordinate axes of the positioner coordinate system O3. The rotation matrix of the entire rotation process is:

[0123]

[0124] It can be seen that R2 is composed of the first rotation matrix of the projector, the second rotation matrix of the projector, the first rotation matrix of the locator, and the third rotation matrix of the locator.

[0125] The matrix transformation from the projector coordinate system O2 to the locator coordinate system O3 can be expressed by the following rotation + translation:

[0126]

[0127] Define the above coordinate conversion matrix as T2, and the projection-positioning space conversion relationship between the projector and the locator is obtained. Among them, (x', y', z') is the coordinate of any point in the projector coordinate system O2, and (x, y, z) is the corresponding coordinate of (x', y', z') in the locator coordinate system O3 after conversion by the conversion matrix T2. Correspondingly, the remote control-positioning space conversion relationship between the remote control and the locator can also be obtained according to this method, that is, the above conversion matrix T1, which will not be repeated here.

[0128] Among them, the purpose of the above steps S11 and S12 is to obtain the remote control-positioning space conversion relationship between the remote control coordinate system and the locator coordinate system, and the purpose of steps S13 and S14 is to obtain the projection-positioning space conversion relationship between the projector coordinate system and the locator coordinate system, wherein the order of steps S11, S12 and steps S13, S14 can be executed in the order of S11→S12→S13→S14, or in the order of S13→S14→S11→S12, or steps S11→S12 and S13→S14 can be executed simultaneously.

[0129] S2: Obtain the projection distance of the projector relative to the target projection surface, and the optical path angle when the projector projects onto the target projection surface, and determine the projection area of ​​the projection image on the target projection surface in the projector coordinate system according to the projection distance and the optical path angle;

[0130] Wherein, step S2 specifically includes:

[0131] S21: Obtaining the tilt angle of the projector relative to the target projection surface and the preset optical-mechanical field angle, and obtaining the optical path angle when the projector projects onto the target projection surface according to the optical-mechanical field angle and the tilt angle;

[0132] S22: Obtaining the projection distance of the projector relative to the target projection surface, and determining the projection area of ​​the projection image on the target projection surface in the projector coordinate system according to the optical path angle and the projection distance.

[0133] In a specific example, the coordinates of the four vertices P1, P2, P3, and P4 of the projection surface in the projector coordinate system O2 can be calculated by the optomechanical FOV (field of view) parameters and the projector automatic trapezoidal algorithm sensor (camera / ToF / gyroscope), where the projector coordinate system O2 is a coordinate system with the optomechanical optical center of the projector as the origin, the y-axis in the projector coordinate system O2 coincides with the optomechanical optical axis of the projector, and the Oxy plane is parallel to the horizontal plane of the projector body. Consider the case where the projector is projecting the target projection surface, such as Fig.11 As shown, the horizontal angle α of the projector optical machine FOV is known F , pitch angle β F , the optical machine is an upward biased optical machine, that is, ∠P5O2P6=β F , ∠P4O2P5=α F / 2=∠P3O2P5. The projection distance of the optical machine is obtained by the distance sensor TOF, which is k, that is, O2P5=k, and the surface equation of the projection surface is y=k. The coordinates of the four vertices in the projector coordinate system O2 are: P1(-k*tan(α F / 2),k,k*tanβ F )、P2(k*tan(α F / 2),k,k*tanβ F )、P3(k*tan(α F / 2),k,0)、P4(-k*tan(α F / 2),k,0). After the projector performs automatic trapezoidal correction, the yaw angle / roll angle / pitch angle centered on the optical axis can be calculated based on the automatic trapezoidal algorithm sensor (camera / ToF / gyroscope). Fig.12As shown, project the x and z axes of the projector coordinate system O2 into the plane of the projection screen and set them as the xˊ and zˊ axes. Rotate the plane of the projection screen by the yaw angle around the zˊ axis and by the flip angle around the xˊ axis to obtain the plane equation of the projection screen after the automatic trapezoid; combine the above data to find the coordinates of the four vertices in the projector coordinate system O2.

[0134] Among them, the order of the above steps S1 and S2 can be executed in the order of S1→S2, or in the order of S2→S1, or steps S1 and S2 can be executed simultaneously, and then S3 is further calculated by combining the parameters obtained in steps S1 and S2.

[0135] S3: Obtain the coordinates of the intersection of the remote control pointing and the projection area according to the conversion relationship between the remote control pointing and the remote control-positioning space in the remote control coordinate system, and the conversion relationship between the projection area and the projection-positioning space;

[0136] In some embodiments, step S3 specifically includes:

[0137] S31: establishing a plane equation corresponding to the projection picture in the locator coordinate system according to the projection area of ​​the projection picture on the target projection surface in the projector coordinate system and the projection-positioning space conversion relationship;

[0138] S32: Obtaining a straight line equation of the designated straight line in the locator coordinate system according to the calculation of the conversion relationship between the designated straight line in the remote control coordinate system and the remote control-positioning space, wherein the straight line equation represents the coordinates of the remote control in the locator coordinate system and the remote control direction;

[0139] S33: Calculate the coordinates of the intersection between the remote control pointing and the projection image in the locator coordinate system according to the plane equation and the line equation;

[0140] S34: Obtain the intersection coordinates between the remote control pointing and the projection screen in the projector coordinate system according to the intersection coordinates between the remote control pointing and the projection screen in the locator coordinate system and the projection-locating space conversion relationship.

[0141] In this embodiment, the relationship between the straight line equation of the remote control pointing and the spatial coordinates of the projection screen is uniformly calculated in the locator coordinate system O3; then the coordinates of the intersection point P' are converted into spatial coordinates under the projector coordinate system O2 through the inverse matrix to obtain the coordinates of point P' under the projection screen; subsequently, the P' coordinates are converted into the screen pixel coordinate system to obtain the position coordinates of the remote control pointing point on the projection screen.

[0142] like Fig.13In a specific example, the equation of the projected image in the projector coordinate system O2 is defined as A2. After conversion by the coordinate transformation matrix T2, the plane equation A3 of A2 in the locator coordinate system O3 can be obtained. The straight line equation of the y-axis (remote control pointing) in the remote control coordinate system O1 is defined as Q1. After conversion by the coordinate transformation matrix T1, the straight line equation Q3 of Q1 in the locator coordinate system O3 can be obtained. After the above two transformations, the remote control pointing straight line Q3 and the projection image plane A3 are normalized to the locator coordinate system O3; the intersection point P' of Q3 and O3 in the locator coordinate system O3 is calculated, and then the coordinates of point P' are transformed through the inverse matrix of the coordinate transformation matrix T2. The point P'2 is converted to the projector coordinate system O2, that is, P'2 is the coordinate of the intersection between the remote control pointing and the projected image in the projector coordinate system O2.

[0143] In other embodiments, the designated straight line of the remote control coordinate system may be directly converted to the projector coordinate system, and the intersection of the two may be further calculated with the plane equation in the projector coordinate system, so that the coordinates of the intersection between the remote control pointing and the projected image in the projector coordinate system can be directly obtained. In this case, step S3 specifically includes:

[0144] S31': establishing a plane equation of the projection picture in the projector coordinate system according to the projection area of ​​the projection picture on the target projection surface in the projector coordinate system;

[0145] S32': Obtain the linear equation of the designated linear map in the projector coordinate system according to the calculation of the designated linear map in the remote control coordinate system and the remote control-positioning space conversion relationship and the projection-positioning space conversion relationship. The linear equation represents the coordinates of the remote control in the projector coordinate system and the remote control direction.

[0146] S33': Calculate the coordinates of the intersection between the remote control pointing and the projected image in the projector coordinate system according to the plane equation and the coordinates and pointing of the remote control in the projector coordinate system.

[0147] S4: Displaying an indication cursor at a position corresponding to the intersection coordinates on the projection screen.

[0148] Finally, the coordinates of point P'2 in the projector coordinate system O2 are converted to the pixel coordinate system of the projector optical screen, so that the coordinates of the intersection between the remote control and the projected screen can be obtained in the projected screen, and the indicator cursor is displayed accordingly; thus, even if the remote control cannot emit a light beam, the position of the projected screen where it points will also be displayed. The indicator cursor is formed by the projection of the projector optical machine, and the indicator cursor can be, for example, a circular light spot or other types of indicator patterns, which are used to reflect the position pointed by the remote control in the projected screen.

[0149] In the scheme of the first embodiment of the present invention, a group of wireless modules and corresponding spatial angle measurement antennas are built into the projector, the remote control, and the locator, respectively, which can improve the measurement accuracy of the remote control position and angle, and further improve the accuracy of the display position of the indicator light spot in the projection screen. If the locator is not provided, and only the antenna and the module are built into the remote control and the projector, this integrated solution requires that the electromagnetic wave beam of the antenna and the module of the projector can cover the hemisphere. Due to the beam field angle and measurement accuracy issues, it is necessary to build multiple groups of antenna arrays and wireless modules into the projector, which is costly and complex in structure. Alternatively, if only the antenna and the wireless module are built into the locator and the remote control, only the spatial angle and distance measurement are performed between the locator and the remote control, and only the spatial pointing relationship between the remote control and the locator is sensed, this solution cannot automatically sense the spatial relationship between the locator and the projection screen, and cannot achieve automatic calibration. The user is required to place the locator at a fixed position such as the bottom edge center or the top edge center of the projection screen, which increases the difficulty of user use and limits the usage scenario. Alternatively, the locator function can be placed in the projection screen, with a built-in antenna and wireless module in the screen. This solution requires the configuration of a specific screen, which also increases the difficulty of user use and limits the usage scenarios.

[0150] Embodiment 2 of the present invention discloses a projector, including a processor and a memory, wherein a computer program is stored in the memory, and the computer program can be read and executed by the processor to implement the interactive method for projecting a picture in embodiment 1.

[0151] The memory is a storage device that stores programs and data executed by the processor in a non-volatile manner. The memory is composed of semiconductor storage elements such as magnetic storage devices, flash read-only memories (ROM), or other types of non-volatile storage devices. The memory may also include random access memories (RAM) that constitute the working area of ​​the processor. The memory stores the control program executed by the processor. The processor may be composed of a single processor or a combination of multiple processing groups. The processor executes the control program to control various parts of the projection device.

[0152] Embodiment 3 of the present invention discloses a storage medium, in which a computer program is stored. The computer program is configured to be executable by a processor to execute the interactive method for projecting a picture in embodiment 1.

[0153] Optionally, the storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0154] The background section of the present invention may contain background information about the problem or environment of the present invention, rather than describing the prior art by others. Therefore, the content contained in the background section is not an admission by the applicant that the prior art is available.

[0155] The above content is a further detailed description of the present invention in combination with specific / preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, without departing from the concept of the present invention, it can also make several substitutions or modifications to these described embodiments, and these substitutions or modifications should be regarded as belonging to the protection scope of the present invention. In the description of this specification, the description of reference terms "an embodiment", "some embodiments", "preferred embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in combination with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, in the absence of mutual contradiction, the technical personnel in this field can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions and modifications can be made herein without departing from the scope defined by the attached claims.

Claims

1. An interactive method for projecting a picture, characterized in that: The following steps are involved: Acquire a first spatial relationship between the remote control and the locator, and obtain a remote control-locating space conversion relationship between a remote control coordinate system of the remote control and a locator coordinate system of the locator according to the first spatial relationship; Acquire a second spatial relationship of the projector relative to the locator, and obtain a projection-positioning space conversion relationship between a projector coordinate system of the projector and a locator coordinate system according to the second spatial relationship; Obtaining a projection distance of the projector relative to a target projection surface and an optical path angle when the projector projects onto the target projection surface, and determining a projection area of ​​the projection image on the target projection surface in the projector coordinate system according to the projection distance and the optical path angle; Obtaining the coordinates of the intersection of the remote control pointing and the projection area according to the conversion relationship between the remote control pointing and the remote control-positioning space in the remote control coordinate system, and the conversion relationship between the projection area and the projection-positioning space; A pointer is displayed at a position corresponding to the intersection coordinates on the projection screen.

2. The interactive method for projecting images according to claim 1, characterized in that: The obtaining of a first spatial relationship between the remote controller and the locator comprises: Acquire phase differences of multiple groups of first wireless ranging signals between the remote control and the locator, posture parameters of the remote control, and posture parameters of the locator, and determine a first spatial relationship between the remote control and the locator according to the phase differences of the multiple groups of first wireless ranging signals, the posture parameters of the remote control, and the posture parameters of the locator; The acquiring of a second spatial relationship of the projector relative to the locator comprises: The phase difference of multiple groups of second wireless ranging signals between the projector and the locator, the posture parameters of the projector and the posture parameters of the locator are obtained, and the second spatial relationship between the projector and the locator is determined according to the phase difference of multiple groups of second wireless ranging signals, the posture parameters of the projector and the posture parameters of the locator.

3. The interactive method for projecting images according to claim 2, characterized in that: The remote controller is provided with a plurality of first UWB antennas, the projector is provided with a plurality of second UWB antennas, and the locator is provided with a third UWB antenna; Respectively acquiring ranging signals from the third UWB antenna through the plurality of the first UWB antennas to obtain a plurality of groups of first wireless ranging signals between the remote controller and the locator; The ranging signals from the third UWB antenna are respectively acquired through the plurality of the second UWB antennas to obtain a plurality of groups of second wireless ranging signals between the projector and the locator.

4. The interactive method for projecting images according to claim 2, characterized in that: The obtaining of the first spatial relationship of the remote control relative to the locator comprises: obtaining the distance between the origin of the remote control coordinate system and the origin of the locator coordinate system, a first vector in the remote control coordinate system, and a second vector in the locator coordinate system according to the phase difference of the plurality of groups of the first wireless ranging signals, and determining the first gravity vector in the remote control coordinate system according to the posture parameters of the remote control, and determining the third gravity vector in the locator coordinate system according to the posture parameters of the locator; The method of obtaining the second spatial relationship of the projector relative to the locator includes: obtaining the distance between the origin of the projector coordinate system and the origin of the locator coordinate system, the third vector in the projector coordinate system and the fourth vector in the locator coordinate system according to the phase difference of multiple groups of the second wireless ranging signals, and determining the second gravity vector in the projector coordinate system according to the posture parameters of the projector, and determining the third gravity vector in the locator coordinate system according to the posture parameters of the locator.

5. The interactive method for projecting images according to claim 4, characterized in that: The step of obtaining the remote control-positioning space conversion relationship between the remote control coordinate system of the remote control and the positioner coordinate system of the positioner according to the first spatial relationship includes: Obtaining a first rotation matrix of the remote control according to an angle between a first gravity vector in the remote control coordinate system and a coordinate axis and a coordinate plane of the remote control coordinate system, wherein the first rotation matrix of the remote control includes a rotation relationship that makes the z-axis in the remote control coordinate system parallel to the first gravity vector; Obtaining a first rotation matrix of the locator according to the angle between the third gravity vector in the locator coordinate system and the coordinate axis and the coordinate plane of the locator coordinate system, wherein the first rotation matrix of the locator includes a rotation relationship that makes the z-axis in the locator coordinate system parallel to the third gravity vector; According to the first vector and the first rotation matrix of the remote control, a first rotation vector is obtained, and according to the angle between the first rotation vector and the coordinate axis of the remote control coordinate system rotated based on the first rotation matrix of the remote control, a second rotation matrix of the remote control is obtained; according to the second vector and the first rotation matrix of the locator, a second rotation vector is obtained, and according to the angle between the second rotation vector and the coordinate axis of the locator coordinate system rotated based on the first rotation matrix of the locator, a second rotation matrix of the locator is obtained; Obtaining a first translation matrix according to the distance between the origin of the remote control coordinate system and the origin of the locator coordinate system, and the angle between the second vector and each coordinate axis of the remote control coordinate system; According to the first rotation matrix of the remote control, the second rotation matrix of the remote control, the first rotation matrix of the locator, the second rotation matrix of the locator and the first translation matrix, a remote control-positioning space conversion relationship between the remote control coordinate system and the locator coordinate system is obtained.

6. The interactive method for projecting images according to claim 4, characterized in that: The step of obtaining the projection-positioning space conversion relationship between the projector coordinate system of the projector and the positioner coordinate system according to the second spatial relationship includes: Obtaining a first rotation matrix of the projector according to an angle between a second gravity vector in the projector coordinate system and a coordinate axis and a coordinate plane of the projector coordinate system, wherein the first rotation matrix of the projector includes a rotation relationship that makes the z-axis in the projector coordinate system parallel to the second gravity vector; Obtaining a first rotation matrix of the locator according to the angle between the third gravity vector in the locator coordinate system and the coordinate axis and the coordinate plane of the locator coordinate system, wherein the first rotation matrix of the locator includes a rotation relationship that makes the z-axis in the locator coordinate system parallel to the third gravity vector; According to the third vector and the first rotation matrix of the projector, a third rotation vector is obtained, and according to the angle between the third rotation vector and the coordinate axis of the projector coordinate system rotated based on the first rotation matrix of the projector, a second rotation matrix of the projector is obtained; according to the fourth vector and the first rotation matrix of the locator, a fourth rotation vector is obtained, and according to the angle between the fourth rotation vector and the coordinate axis of the locator coordinate system rotated based on the first rotation matrix of the locator, a third rotation matrix of the locator is obtained; A second translation matrix is ​​obtained according to the distance between the origin of the projector coordinate system and the origin of the locator coordinate system, and the angle between the fourth vector and each coordinate axis of the projector coordinate system. According to the first rotation matrix of the projector, the second rotation matrix of the projector, the first rotation matrix of the locator, the third rotation matrix of the locator and the second translation matrix, a projection-positioning space conversion relationship between the projector and the locator is obtained.

7. The interactive method for projecting images according to claim 1, characterized in that: The acquiring of the projection distance of the projector relative to the target projection surface and the optical path angle when the projector projects onto the target projection surface, and determining the projection area of ​​the projection picture on the target projection surface in the projector coordinate system according to the projection distance and the optical path angle comprises: Obtaining the inclination angle of the projector relative to the target projection surface and the preset optical-mechanical field angle, and obtaining the optical path angle when the projector projects onto the target projection surface according to the optical-mechanical field angle and the inclination angle; The projection distance of the projector relative to the target projection surface is obtained, and the projection area of ​​the projection picture on the target projection surface in the projector coordinate system is determined according to the optical path angle and the projection distance.

8. The interactive method for projecting images according to claim 1, characterized in that: The obtaining the coordinates of the intersection of the remote control pointing and the projection area according to the conversion relationship between the remote control pointing and the remote control-positioning space in the remote control coordinate system and the conversion relationship between the projection area and the projection-positioning space comprises: According to the projection area of ​​the projection picture on the target projection surface in the projector coordinate system and the projection-positioning space conversion relationship, establish the plane equation corresponding to the projection picture in the locator coordinate system; according to the operation of the designated straight line of the remote control coordinate system and the remote control-positioning space conversion relationship, obtain the straight line equation mapped by the designated straight line in the locator coordinate system, and the straight line equation represents the coordinates of the remote control in the locator coordinate system and the remote control pointing; according to the plane equation and the straight line equation, calculate the intersection coordinates between the remote control pointing and the projection picture in the locator coordinate system; according to the intersection coordinates between the remote control pointing and the projection picture in the locator coordinate system and the projection-positioning space conversion relationship, obtain the intersection coordinates between the remote control pointing and the projection picture in the projector coordinate system; or, According to the projection area of ​​the projection picture on the target projection surface in the projector coordinate system, the plane equation of the projection picture in the projector coordinate system is established; according to the calculation of the designated straight line of the remote control coordinate system and the remote control-positioning space conversion relationship and the projection-positioning space conversion relationship, the straight line equation of the designated straight line mapped in the projector coordinate system is obtained, and the straight line equation represents the coordinates of the remote control in the projector coordinate system and the remote control pointing; according to the plane equation and the coordinates and pointing of the remote control in the projector coordinate system, the coordinates of the intersection between the remote control pointing and the projection picture in the projector coordinate system are calculated.

9. A projector, characterized in that: It comprises a processor and a memory, wherein a computer program is stored in the memory, and the computer program can be read by the processor to be executed to implement the interactive method for projecting pictures as described in any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to be executable by a processor to execute the interactive method for projecting images as described in any one of claims 1 to 8.

Images