Relative pointing device and method and interaction device

Abstract

The invention discloses a relative pointing device and method and an interaction device. In order to solve the problems of high power consumption, low real-time performance and high cost in an existing scheme, the relative pointing method comprises the steps that a DVS is arranged in pointing equipment and used for sensing flickering light sources arranged on pointed equipment, the pointed equipment is provided with at least two flickering light sources, and corresponding imaging points are formed in the DVS; and according to the geometric center of the imaging point, the DVS resolution and the resolution of the pointed device, mapping the geometric center to the pointed device to obtain a mapping point. The invention has the advantages of low cost, low power consumption and high real-time performance, and can realize the relative pointing of the user to the pointed equipment.

Description

Technical Field

[0001] The present invention relates to the field of information interaction, and in particular to a relative pointing device and method, and an interaction device. Background Art

[0002] In the field of home entertainment screen devices, screen-directed interaction is a new interaction mode that breaks through the limitation of traditional remote control solutions that can only jump to the selection through buttons. When the user points to the screen with the remote control or related equipment, the interaction effect corresponding to the position pointed by the remote control can be presented on the screen with the help of specific technical solutions.

[0003] The current solutions are mainly divided into categories such as Ultra-Wide Band (UWB), gyroscope air mouse and under-screen optical sensor. Among them, UWB solutions are more common in the current market, and corresponding products have been launched.

[0004] With the help of multiple antennas and UWB's precise positioning technology, the UWB signal transceiver function is set on the screen and remote control. However, this solution has the disadvantages of high cost, high power consumption load on the remote control (due to the radar transceiver function) and poor real-time performance.

[0005] In the gyroscope / IMU air mouse solution, the gyroscope is a device that can sense posture changes, is independent of position, and can detect the angular velocity of rotation on each axis of an object during motion. The gyroscope air mouse uses the built-in gyroscope to sense the movement and rotation of the user's hand in three-dimensional space in real time, and converts these movements into movement instructions for the cursor on the screen, thereby achieving control of the screen cursor. The main defect of this type of solution is that due to the characteristics of the differential sensor, its perception data will drift, resulting in poor accuracy and stability, and high user learning costs.

[0006] The under-screen sensor solution requires a large number of light sensor arrays to be placed under the display screen, and a laser device to be installed on the remote control. The under-screen sensor senses the position of the sensor on the screen triggered by the laser to determine the direction of the remote control. Although this solution has extremely high pointing accuracy, the cost is too high to achieve mass production and large-scale application.

[0007] There are other processing solutions, such as using image sensors to identify TVs and perform reverse SLAM. This method requires processing images on the camera side, and determining the remote control direction through visual calibration and inverse calculation. This solution requires high computing power on the camera side (i.e., remote control). Since the reporting rate of smooth pointing usually needs to be greater than 120Hz, setting up an image processor on the remote control side and achieving 120 frames / second image processing will lead to high costs and power consumption issues.

[0008] Based on this, there is an urgent need in this field to provide a low-cost, low-power, and high-real-time solution for relative pointing remote control solutions. Summary of the invention

[0009] In order to alleviate or partially alleviate the above technical problems, the solution of the present invention is as follows:

[0010] On the one hand, the present invention discloses a relative pointing method, in which a DVS is equipped in a pointing device and is used to sense a flashing light source arranged on a pointed device, wherein the pointed device is provided with at least two flashing light sources and forms a corresponding imaging point in the DVS; according to the geometric center of the imaging point and the resolution of the DVS and the resolution of the pointed device, the geometric center is mapped to the pointed device to obtain a mapping point.

[0011] Furthermore, the number of the scintillation light sources is 2, and the geometric center is the midpoint of the two imaging points.

[0012] Furthermore, according to information output by a sensor in the pointing device for determining the posture, a rotation matrix about the current posture of the pointing device is obtained; and according to the rotation matrix, the mapping point is corrected to obtain the mapping point after rotation correction.

[0013] Furthermore, the correcting the mapping points according to the rotation matrix refers to: applying a transposed matrix of the rotation matrix to the mapping points.

[0014] Furthermore, the event stream output by the DVS is filtered to extract imaging points associated with the flickering light source.

[0015] Furthermore, after or during filtering the event stream output by the DVS, the identifier of each flashing light source is identified.

[0016] Furthermore, the processing step of obtaining the mapping point after rotation correction according to the event stream output by the DVS occurs in the pointing device or the pointed device, or occurs partly in the pointing device and partly in the pointed device; the pointed device executes a corresponding response according to the mapping point after rotation correction.

[0017] On the other hand, the present invention discloses a pointing device: a DVS is provided in the pointing device and is used to sense a flashing light source arranged on a pointed device, wherein the pointed device is provided with at least two flashing light sources and forms a corresponding imaging point in the DVS; in the pointing device, the following processing steps are performed: according to the geometric center of the imaging point and the DVS resolution and the resolution of the pointed device, the geometric center is mapped to the pointed device to obtain a mapping point.

[0018] Furthermore, the pointing device also includes a sensor for determining posture; in the pointing device, the following processing steps are also performed: based on the information output by the sensor in the pointing device for determining posture, a rotation matrix about the current posture of the pointing device is obtained; the mapping point is corrected according to the rotation matrix to obtain the mapping point after rotation correction.

[0019] Furthermore, the sensor for determining the position and posture is an IMU.

[0020] Furthermore, the pointing device offloads part or all of the processing steps to the pointed device.

[0021] On the other hand, the present invention discloses an interactive device, comprising a pointing device and a pointed device: the pointing device and the pointed device are provided with a DVS in the pointing device, and are used to sense a flickering light source arranged on the pointed device; wherein at least two flickering light sources are arranged on the pointed device, and corresponding imaging points are formed in the DVS; in the pointing device and / or the pointed device, the following processing steps are performed: filtering the event stream output by the DVS, extracting imaging points associated with the flickering light sources; mapping the geometric center to the pointed device according to the geometric center of the imaging point and the resolution of the DVS and the resolution of the pointed device, to obtain a mapping point.

[0022] Furthermore, in the pointing device and / or the pointed device, the following processing steps are also performed: obtaining a rotation matrix about the current pointing device posture based on the information output by the IMU in the pointing device; correcting the mapping points based on the rotation matrix to obtain the mapping points after rotation correction.

[0023] The technical solution of the present invention has one or more of the following beneficial technical effects:

[0024] (1) Low cost advantage;

[0025] (2) It has the advantage of low power consumption;

[0026] (3) It has the advantage of high real-time performance;

[0027] (4) The user uses the pointing device to achieve relative pointing to the pointed device.

[0028] In addition, other beneficial effects of the present invention will be mentioned in the specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a schematic diagram of an embodiment of a screen pointing device with relative pointing according to the present invention;

[0030] Figure 2This is a sample diagram of the event stream output by DVS after frame compression;

[0031] Figure 3 It is a schematic diagram of a driving signal waveform of a light source carrying coded information;

[0032] Figure 4 It is a schematic diagram of a user operating a pointing device to move downward;

[0033] Figure 5 It is a schematic diagram of imaging under rotation. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] In order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order.

[0036] In the present invention, the term "relative pointing" describes the relative movement trend of the pointing device controlled by the user relative to the pointed device. In the pointed device, there is usually a mapping point reflecting the current relative pointing direction of the pointing device, and the movement of the mapping point is consistent with the movement trend of the pointing device. In contrast, absolute pointing is when the intersection of the mapping point, the camera optical axis and the plane where the flashing light source is located is basically coincident.

[0037] For example, the movement trend of the mouse controlled by the user is consistent with the movement trend of the cursor on the screen, but the movement distance of the cursor on the screen is usually not strictly equal to the movement distance of the mouse. A shorter mouse stroke can correspond to a larger cursor movement distance, which is relative pointing.

[0038] The term "pointing device" refers to a device controlled by a user to indicate or respond to the user's selection / confirmation / action intention. It is usually held by the user and reflects the user's operation intention or movement trend in real time by aiming the pointing device at the pointed device and moving or rotating the pointing device.

[0039] The term "relative pointing device" is a type of pointing device and has a relative pointing characteristic. The "pointing device" in the present invention usually refers specifically to a relative pointing device.

[0040] The term "pointed device" refers to a device that a pointing device can point to, including a device that transmits information to a user to achieve the purpose of interaction with the user, such as a projector screen, a TV screen, or other devices. In the present invention, the pointed device includes not only a device that transmits information to a user, but also an information processing system associated therewith, such as a projector screen and a projector body, which together as a whole become the pointed device.

[0041] The pointed device can eventually obtain information reflecting the user's operating intention or movement trend to determine the next operation or control instruction. For example, when the user slides the pointing device left / right, the pointed device and its associated information processing system are instructed to switch to play the next video.

[0042] The term "flickering light source" refers to a light source (such as a light emitting diode) that is continuously turned on or off by a driving signal, thereby achieving a continuous flickering of the light source. The duty cycle of the driving signal can be designed to be different parameters (not necessarily just 50%), and can also carry coded information, so that the flickering light source can convey the coded information.

[0043] The term "filtering" refers to screening or identifying (pulse) events in an event stream by certain rules or means to retain or filter out some or all events in the event stream. For example, filtering can be performed based on the frequency or period of occurrence of events, or can be performed based on whether they can be decoded or whether the decoding results meet expectations.

[0044] The term "DVS" or "event camera" or "dynamic vision sensor" or "dynamic event camera" refers to a device that includes dynamic vision pixels that can output event signals (referred to as events) generated by changes in light intensity. The device may also include other types of sensors and / or processing components. Dynamic vision pixels sense changes in light intensity, and when the change in light intensity (brightening or darkening) in the sensing area exceeds a certain range, an event is generated.

[0045] The term "imaging point" refers to a series of events generated by a scintillation light source during the imaging process in DVS. Each event includes the pixel coordinates that generate the event. The pixel coordinates among these events that best represent or reflect the current position of the scintillation light source are called the pixel coordinates or imaging points associated with the scintillation light source.

[0046] Figure 1 A schematic diagram showing an embodiment of a screen pointing device with relative pointing according to the present invention.

[0047] In this embodiment, the pointed device may be a television set, and a flashing light source may be provided at each of two corners of the television set. The two corners may be the two lower corners of the television set or the two upper corners.

[0048] The flashing light source may be located at not only the corners of the pointed device but also non-corners, such as the midpoints of two vertical sides. The present invention is not limited to a specific layout.

[0049] In the user's pointing device, the pointing device includes a dynamic vision sensor (DVS), which is also called an event camera. The pointing device in the present invention generally refers to a relative pointing device.

[0050] When the user operates the pointing device, which is usually called a remote control in this embodiment, the pointed device appears in the field of view of the DVS when the user uses it. When the user operates the pointing device, since the pointing device has movements such as rotation and translation, the edges or textures of all objects in the field of view of the DVS usually generate pulse events, and several pulse events constitute an event stream.

[0051] In addition, preferably, the pointing device in the present invention further includes a sensor for determining the position and posture. For example, the sensor for determining the position and posture is an inertial measurement unit (IMU). In one embodiment, the IMU can output or obtain a quaternion (w, x, y, z) to represent the posture, which can smoothly represent the rotation of the object, wherein the rotation axis Rotation angle θ=2arccos(w). In the present invention, how to use the output information of the IMU will be described later.

[0052] Figure 2 This figure shows an example of the event stream output by DVS after frame compression. Since the surroundings of the pointed device usually include various objects, such as coffee tables, ornaments, etc., these objects may be imaged in DVS, generating a large number of events. In addition, the flickering light source in the pointed device will also be imaged in DVS due to its continuous high-frequency flickering. Usually, several different events in the event stream are distributed at different time points, but by compressing the events within a period of time on a two-dimensional plane, a picture similar to a traditional frame image can be obtained.

[0053] In the present invention, the frequency of the high-frequency flickering flashing light source may be 1200 Hz, 1000 Hz, 900 Hz, 500 Hz, 400 Hz, etc. The duty cycle of the driving signal controlling the flashing light source may be 20%, 50%, 75%, etc., for example.

[0054] After the DVS senses the changes in light and darkness in these fields of view, it outputs an event stream. In one embodiment, the event stream output by the DVS can be directly sent to the pointed device, which usually has large computing resources, and then the pointed device performs filtering operations on the event stream output by the DVS, and subsequently obtains the mapping point in the pointed device.

[0055] In one type of embodiment, the event stream output by the DVS performs a filtering operation in the pointing device, and subsequently obtains an operation of mapping points in the pointed device.

[0056] In one type of embodiment, the first part (such as performing filtering operations) of the processing of the event stream output by the DVS occurs in the pointing device, and the second part of the processing operation occurs in the pointed device.

[0057] Furthermore, the present invention performs a filtering operation on the event stream generated by the DVS, filtering out events generated by non-flickering light sources and retaining events generated by flickering light sources. Specifically, by matching the flickering pattern of the flickering light source, only events generated by the flickering pattern of the light source are retained during filtering.

[0058] For example, Spiking Neural Networks (SNN) can be used to implement filtering, and the event stream can be filtered based on bandpass filtering, high-pass filtering, and similarity filtering methods to retain events generated by flickering light sources. These filtering methods can be based on event time interval schemes or SNN schemes.

[0059] For example, some feasible filtering methods can refer to the following Chinese patent application documents: CN116757256A, CN116703971A, CN116579396A, which are all incorporated into this application by reference. Regarding the filtering operation, the present invention is not limited to this and is not limited to this.

[0060] Even though DVS outputs a large number of events, most of the events generated by the edges or textures of objects in the field of view can be filtered out compared to events generated by high-frequency flickering light sources, which facilitates subsequent rapid information processing.

[0061] Based on this fact, in one class of embodiments, the pointing device can send the filtered event stream to the pointed device, and then the pointed device obtains the mapping points in the pointed device, which reduces the data transmission bandwidth requirements between the pointing device and the pointed device, reduces the power consumption required for the pointing device to send data, and avoids directly obtaining the mapping points in the pointed device in the pointing device. In other words, the processing steps for obtaining the mapping points in the pointed device according to the event stream output by the DVS occur in the pointed device, or partly in the pointing device and partly in the pointed device, which is to offload the computing task to the pointed device with stronger computing power. The offloading here refers to transmitting the obtained intermediate processing results to the pointed device (usually a wireless signal), and the pointed device calculates the final mapping points in the pointed device.

[0062] Figure 3 The waveform of a driving signal of a light source carrying coded information is shown, which drives the light source to produce a specific pattern of flickering. The coded information is transmitted through the channel with light as the carrier and reaches the receiving side DVS. The DVS generates an event based on the received optical signal and adds it to the event stream.

[0063] After receiving the event stream, decoding is performed within the predetermined decoding cycle. In this example Figure 3 The decoding result is "0011". Events that cannot be decoded or whose decoding results are invalid codes can be regarded as noise and filtered out.

[0064] In the present invention, optionally, after the aforementioned filtering operation, the identifier of the flickering light source associated with the retained event can be identified according to the retained event. In the DVS imaging process, due to factors such as rotation, translation, and lens distortion of the camera lens, the two-dimensional imaging in the camera cannot fully and accurately reflect the geometric distribution in the world coordinate system.

[0065] For the scheme of implementing filtering through decoding operation, since the identification information of each flashing light source can be directly encoded during the encoding process, such as binary encoding, the corresponding identification information can be directly obtained during decoding, thereby achieving the purpose of identifying the flashing light source.

[0066] For example, in the embodiment where the decoding result is "0011", under the binary coding rule, the corresponding flashing light source identifier is 3. For other flashing light sources, they can be encoded as other information, such as "0001", "0010", etc. Through different identifiers, it is possible to filter and identify which events are generated by which flashing light source in the event stream obtained from the DVS output.

[0067] For filtering schemes such as high-pass filtering and band-pass filtering, in the example, the identity of each flickering light source can be identified based on the geometric relationship of the imaging position of the flickering light source in the imaging plane, the continuity in time, the frequency characteristics, etc.

[0068] For example, in the embodiment of two flashing light sources, the events retained after filtering are stored in two different queues representing two different flashing light sources, and premature historical events can be removed from the queues. For the newly retained events, according to the distance relationship between the pixel coordinates of the event and the pixel coordinates of the events in the two queues, the event is selected to be stored in the queue with a closer pixel coordinate distance.

[0069] For another example, in an embodiment with more than two flickering light sources, since the distances of each flickering light source from the center of gravity of the flickering light source remain unchanged and present a fixed distance order, the distance order generally does not change in the pixel plane after DVS perception. The calculated distances can be ordered, and the flickering light source corresponding to the pixel coordinates associated with each flickering light source can be identified based on the ordering result.

[0070] Although the present invention has introduced different flashing light source identification methods above, the present invention is not limited thereto and is not intended to be limited thereto.

[0071] Figure 4 This is a schematic diagram of a user operating a pointing device to move downward. Events in the event stream carry pixel coordinate information that generates the event. In a short period of time, a flickering light source usually causes pixels in a small area in the DVS to generate events. For example, the average coordinate value of pixels or events in each small area can be used as the coordinate of the flickering light source in the pixel plane, that is, the pixel coordinate or imaging point associated with the flickering light source.

[0072] In the example, the user turns the pointing device downward, and the imaging points of the flickering light sources A and B in the pixel plane of the DVS are a and b respectively, and the midpoint of the imaging point a and the imaging point b is the virtual imaging point c (there is no flickering light source at the midpoint of the actual flickering light sources A and B).

[0073] As the user operates, the field of view of the DVS moves downward, and the imaging points of the scintillation light sources A and B in the pixel plane also change or move accordingly. The midpoint of the scintillation light sources A and B, the virtual imaging point c in the pixel plane, also changes or moves accordingly. Among them, the imaging point a and the imaging point b can be the geometric center of several coordinate points in the event set generated by the scintillation light sources A and B, such as the aforementioned coordinate average solution.

[0074] In the present invention, preferably, the virtual imaging point c, i.e., the flickering light source should be set as the geometric center in the pixel plane, and the mapping point P in the pointed device is used as the final mapping point in the pointed device. The mapping point P is the result of the user's relative pointing and moves in the pointed device with the user's operation.

[0075] Assuming that the resolution of the pointed device is W×H and the resolution of the pixel plane is w×h, as a possible example, the following mapping relationship f between the (virtual or non-virtual) imaging point in the pixel plane and the mapping point in the pointed device can be established:

[0076] f(x i ,y i )=(x i ×W / w,y i ×H / h);

[0077] Where (x i ,y i ) is the coordinate of an imaging point in the pixel plane. Therefore, for the coordinate of the virtual imaging point c The coordinates of the mapping point P in the pointed device are P(x, y) = (x c ×W / w,y c ×H / h), where (x a ,y a )、(x b ,y b ) are the coordinates of imaging point a and imaging point b in the pixel plane respectively.

[0078] Although the above uses two flickering light sources and their midpoints as an example, and the advantage of this type of embodiment is that the modification or design of the pointed device can be minimized, and the cost is the lowest. In fact, for more than two flickering light sources, their geometric center (orthocenter, center of gravity, etc.) in the pixel plane can be used as a virtual imaging point and mapped to the pointed device. For example, in an embodiment, three flickering light sources are included, and the center of gravity of their imaging points in the pixel plane can be taken as the virtual imaging point and mapped to the pointed device.

[0079] The above describes the relative pointing mapping of the camera or pointing device without rotation. However, when the user actually uses it, the pointed device may be turned over or rotated. Therefore, the present invention further considers this situation.

[0080] Figure 5 is a schematic diagram of imaging under rotation. In the case of no rotation, assuming that the coordinate system of the camera is {C 0}, in the case of rotation, the camera's coordinate system is {C}.

[0081] According to the current flashing light sources A and B, the virtual imaging point c in the DVS is obtained, and according to the mapping relationship f, the coordinate system {C 0}, the mapping point of the virtual imaging point c is P c0 ; In the rotating coordinate system {C}, the mapping point of the virtual imaging point c is P c .

[0082] According to the coordinate transformation:

[0083]

[0084] in, represents the coordinate system without rotation {C 0} to the rotation matrix of the coordinate system {C}.

[0085] In order to obtain the unrotated virtual imaging point c, the mapping point is P c0 , applying matrix transformation we get:

[0086]

[0087] In other words, we only need to rotate the mapping point P in the coordinate system c application You can get the mapping point P in the unrotated state c0 .

[0088] The pointing device of the present invention includes an IMU, the information output by which can be used to obtain the rotation matrix Or a matrix related to the rotation matrix, such as

[0089] In this field, the quaternion of the attitude of the current camera (which may have been rotated) relative to the initial state can be obtained by combining IMU data and fusion algorithm. Then according to the quaternion Convert to rotation matrix These are well-known technologies to those skilled in the art. Therefore, the present invention obtains the rotation matrix through the IMU, and further obtains the mapping point P before / after rotation. c0 , which verifies that the present invention can solve the problem of accurately obtaining the coordinate system when the user rotates the pointing device. 0}The mapping point P in c0 , avoiding this adverse effect in real scenarios.

[0090] An IMU typically contains a three-axis accelerometer, a three-axis gyroscope, and an optional three-axis magnetometer. The accelerometer measures acceleration, the gyroscope measures angular velocity, and can track attitude changes based on time integration. The optional magnetometer measures the direction of the Earth's magnetic field and provides a global reference direction.

[0091] As a further introduction to an embodiment of the IMU, after the pointing device is started, it is assumed that the pointing device is in a non-rotated reference posture. Further, the accelerometer is used to determine the gravity / vertical direction, and the magnetometer can be further used to determine the reference horizontal plane direction.

[0092] According to the constructed initial coordinate system, the initial quaternion about the attitude is calculated. For an IMU without a magnetometer, the initial attitude can be set to a unit quaternion (1, 0, 0, 0) using the accelerometer and the known chef camera orientation.

[0093] After that, in each IMU update cycle, the new angular velocity ω(t) is read in and the quaternion is updated: in:

[0094]

[0095] After discretization, it is:

[0096]

[0097] After the update, q is further normalized. To reduce drift, Madgwick or Mahony filters can be used for data fusion, and the measurements of the accelerometer (and magnetometer) are used as observations to correct the quaternion. Through this process, the camera's attitude quaternion relative to the initial state at the current time t can be continuously obtained.

[0098] For a quaternion q=(q w ,q x ,q y ,q z ), the corresponding rotation matrix R(q) is:

[0099]

[0100] That is to say, the camera's attitude quaternion relative to the initial state at the current time t is The corresponding rotation matrix is ​​easily obtained And for the mapping point P in the rotation coordinate system c application You can get the mapping point P in the unrotated state c0 , to achieve correct mapping of relative pointing.

[0101] The above calculation method can be executed by a processor in the pointing device or by the pointed device, and the present invention is not limited to a specific execution location. c0The mapping point P in the unrotated state obtained by the pointed device can be calculated locally on the pointing device according to the event stream output by the DVS. c0 The event stream output by DVS, the event stream after filtering operation, or the mapping point P obtained in the unrotated state can be c0 Any intermediate calculation results in are sent to the pointed device, and the pointed device calculates the final mapping point P in the unrotated state. c0 , which is also the mapping point after rotation correction.

[0102] Finally, the present invention also discloses an interactive device, including a pointing device and a pointed device, and any one of the aforementioned relative pointing methods is applied to the pointing device and / or the pointed device.

[0103] Furthermore, the pointing device is a remote controller, and the pointed device is a television, or a projector and a screen.

[0104] In order to better illustrate the present invention, numerous specific details are provided in the above specific embodiments. It should be understood by those skilled in the art that the present invention can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present invention.

[0105] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A relative pointing method, characterized in that: include: The pointing device is equipped with a DVS, and is used to sense the flashing light source arranged on the pointed device, wherein the pointed device is provided with at least two flashing light sources, and corresponding imaging points are formed in the DVS; According to the geometric center of the imaging point, the DVS resolution, and the resolution of the pointed device, the geometric center is mapped to the pointed device to obtain a mapping point.

2. The relative pointing method according to claim 1, characterized in that: The number of the scintillation light sources is 2, and the geometric center is the midpoint of the two imaging points.

3. The relative pointing method according to claim 1 or 2, characterized in that: Obtaining a rotation matrix about the current posture of the pointing device according to information output by a sensor in the pointing device for determining the posture; The mapping point is corrected according to the rotation matrix to obtain the mapping point after rotation correction.

4. The relative pointing method according to claim 3, characterized in that: The correction of the mapping point according to the rotation matrix refers to: A transposed matrix of the rotation matrix is ​​applied to the mapped points.

5. The relative pointing method according to claim 1, 2 or 4, characterized in that: The event stream output by DVS is filtered to extract the imaging points associated with the flickering light source.

6. The relative pointing method according to claim 5, characterized in that: After or while filtering the event stream output by the DVS, the identity of each flashing light source is identified.

7. The relative pointing method according to claim 3, characterized in that: The processing step of obtaining the mapping point after rotation correction according to the event stream output by the DVS occurs in the pointing device or the pointed device, or partially occurs in the pointing device and partially occurs in the pointed device; The pointed device performs a corresponding response according to the rotationally corrected mapping point.

8. A pointing device, characterized in that: The pointing device is equipped with a DVS, and is used to sense the flashing light source arranged on the pointed device, wherein the pointed device is provided with at least two flashing light sources, and corresponding imaging points are formed in the DVS; In the pointing device, the following processing steps are performed: According to the geometric center of the imaging point, the DVS resolution, and the resolution of the pointed device, the geometric center is mapped to the pointed device to obtain a mapping point.

9. The pointing device according to claim 8, characterized in that: The pointing device also includes a sensor for determining a posture; In the pointing device, the following processing steps are also performed: Obtaining a rotation matrix about the current posture of the pointing device according to information output by a sensor in the pointing device for determining the posture; The mapping point is corrected according to the rotation matrix to obtain the mapping point after rotation correction.

10. An interactive device, comprising a pointing device and a pointed device, characterized in that: The pointing device is equipped with a DVS, and is used to sense the flashing light source set on the pointed device; The pointed device is provided with at least two flashing light sources, and corresponding imaging points are formed in the DVS; In the pointing device and / or the pointed device, the following processing steps are performed: Filter the event stream output by DVS to extract the imaging points associated with the flickering light source; According to the geometric center of the imaging point, the DVS resolution, and the resolution of the pointed device, the geometric center is mapped to the pointed device to obtain a mapping point.

Images