Interaction device and method based on event camera, electronic equipment and storage medium

Abstract

The invention relates to an event camera-based interaction device and method, electronic equipment and a computer readable storage medium, and the device comprises a matched light source which is arranged on display equipment and emits light with the intensity changing along with time; the main controller comprises an event camera and a main control module, and the event camera is used for moving relative to the display equipment and collecting event images of the matched light source when moving relative to the display equipment; and the main control module is used for determining the position or the motion trail of the event camera relative to the display equipment according to the event image, and is also used for performing interaction control on the display equipment according to the position or the motion trail. The interaction function can be expanded, and the interaction efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of interactive control technology, and in particular to an interactive device, method, electronic device and storage medium (computer-readable storage medium) based on an event camera. Background Art

[0002] In daily life and work, display devices such as televisions, computers, and projectors are widely used, and usually require the use of matching interactive devices such as remote controls, mice, or laser pens for indication or control.

[0003] Regardless of the interaction method, the supported interactive functions are very limited.

[0004] For example, laser pens only support operations such as clicking and dragging, and the interactive functions they can support are very limited, far from meeting user interaction needs. Summary of the invention

[0005] The embodiments of the present application provide an interactive device based on an event camera, a control method based on an event camera, an electronic device, and a computer-readable storage medium, aiming to expand interactive functions and improve interactive efficiency.

[0006] In order to achieve the above-mentioned object, according to a first aspect of the present application, an interactive device based on an event camera is provided, which is used to interact with a display device, comprising:

[0007] a matching light source, arranged on the display device, emitting light whose intensity varies with time; and

[0008] The main controller includes an event camera and a main control module, wherein:

[0009] The event camera is used to move relative to the display device, and is also used to capture event images of the supporting light source when moving relative to the display device;

[0010] The main control module is used to determine the position or motion trajectory of the event camera relative to the display device according to the event image, and is also used to interactively control the display device according to the position or the motion trajectory.

[0011] Optionally, the matching light source is arranged on the frame of the display device;

[0012] The interactively controlling the display device according to the position or the motion trajectory includes:

[0013] The optical center of the event camera is mapped onto the display device as a display cursor according to the position, the display cursor is moved according to the motion trajectory, and the display device is interactively controlled based on the display cursor.

[0014] Optionally, the interactively controlling the display device according to the position or the motion trajectory includes:

[0015] Acquiring an operation instruction according to the position or the motion trajectory;

[0016] The display device is interactively controlled according to the operation instruction.

[0017] Optionally, a plurality of function buttons are provided on the main controller, and the function buttons include a control mode switching button, and the control mode switching button is used to trigger a control mode selection event to interactively control the display device according to the control mode corresponding to the control mode selection event.

[0018] According to a second aspect of the present application, a control method based on an event camera is provided, the method being applied to a main controller, the main controller comprising the event camera, the main controller being communicatively connected to a display device, the method comprising:

[0019] Collecting, by the event camera, an event image of a matching light source when the event camera moves relative to the display device, wherein the matching light source is arranged on the display device and emits light whose intensity varies with time;

[0020] The display device is interactively controlled according to the event image.

[0021] Optionally, the interactively controlling the display device according to the event image includes:

[0022] Determine the position or motion trajectory of the event camera relative to the display device according to the event image;

[0023] The optical center of the event camera is mapped onto the display device as a display cursor according to the position, the display cursor is moved according to the motion trajectory, and the display device is interactively controlled based on the display cursor.

[0024] Optionally, the interactively controlling the display device according to the event image includes:

[0025] Determine the position or motion trajectory of the event camera relative to the display device according to the event image;

[0026] Acquiring an operation instruction according to the position or the motion trajectory;

[0027] The display device is interactively controlled according to the operation instruction.

[0028] Optionally, before collecting, by the event camera, an event image of the supporting light source when the event camera moves relative to the display device, the method further includes:

[0029] Trigger control mode selection event;

[0030] Responding to the control mode selection event, so that the main controller enters the target control mode and performs corresponding interactive control;

[0031] Wherein, if the target control mode is the first control mode, then during interactive control, a function control signal is first triggered, and then the function control signal is responded to, a corresponding operation instruction is generated and the operation instruction is sent to the display device to control the display device;

[0032] If the target control mode is the second control mode, then during interactive control, the supporting light source is started so that the supporting light source emits invisible light whose intensity varies with time, and the event camera is started to capture an event image of the supporting light source when the event camera moves relative to the display device.

[0033] According to a third aspect of the present application, an electronic device is provided, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the above-mentioned event-based camera control method.

[0034] According to a fourth aspect of the present application, a computer-readable storage medium is provided, comprising a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the steps of the above-mentioned event-based camera control method.

[0035] According to the fifth aspect of the present application, a computer program product is provided, comprising a computer program, wherein the computer program is stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs the steps of the above-mentioned event-based camera control method.

[0036] In the interactive device, method, electronic device, computer-readable storage medium and computer program product based on the event camera of the embodiment of the present application, when the user moves the main controller to drive the event camera, the event image of the matching light source on the display device is collected by the event camera, thereby, the computing control module can interactively control the display device according to the event image, and since the movement trajectory or action of the main controller relative to the display device varies, the present application can perform a variety of complex operations on the display device, greatly expanding the interactive function. On this basis, since the event camera has a small amount of data, low power consumption, and a low risk of leaking user privacy, the present application can also ensure the security of interactive data.

[0037] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0039] In order to more completely understand the present application and its beneficial effects, the following description will be given in conjunction with the accompanying drawings, wherein the same figure numbers represent the same parts in the following description.

[0040] Figure 1 is a schematic diagram of the structure of an interactive device provided in an exemplary embodiment of the present application;

[0041] Figure 2 is a display interaction schematic diagram provided in an exemplary embodiment of the present application;

[0042] Figure 3 This is a schematic diagram of the light emission frequency of a matching light source provided in an exemplary embodiment of the present application;

[0043] Figure 4 is a first flow chart of a control method provided in an exemplary embodiment of the present application;

[0044] Figure 5 is a second flow chart of a control method provided in an exemplary embodiment of the present application;

[0045] Figure 6 It is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0047] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0048] In the embodiment of the present application, "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / ", unless otherwise specified, generally indicates that the associated objects before and after are in an "or" relationship.

[0049] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to implement and use the invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the invention can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0050] In conjunction with the background technology description of this application, in daily life and work, display devices such as televisions, computers, and projectors are widely used, and usually require the use of matching interactive devices such as remote controls, mice, or laser pens for indication or control.

[0051] However, the existing interactive devices have at least the following defects:

[0052] (1) The remote control is easy to operate and suitable for long-distance control. It generally has basic functions such as channel switching, volume adjustment, and program selection. However, due to the number of buttons and functional design, it is often simple in function and low in flexibility. It cannot support complex display screen operations such as text input and fine graphic editing.

[0053] (2) The mouse has high precision and flexibility, and can achieve precise positioning and complex operations on the display screen, such as clicking, dragging, scrolling, and selecting. However, the interaction between the mouse and the display device (such as a computer) requires a flat operating environment (such as a flat desktop or other operating platform) for stable control of the mouse, which makes the mouse user easily restricted to the limited space next to the desktop;

[0054] (3) Laser pens use laser beams to point to specific areas on the display screen to attract the audience's attention. Some laser pens also have integrated remote control functions that can be used for simple display screen operations, such as page scrolling. However, the display screen operations that laser pens can support are limited. For example, they can only support simple operations such as clicking and dragging, and the laser beam may cause damage to the user's eyes;

[0055] (4) To improve the user experience, a CMOS image sensor can be used to capture the motion trajectory of a remote control or laser pen, locate the optical center of the display screen, and cooperate with specific software to accurately implement remote tapping, dragging, and selection functions on the display screen. However, traditional image sensor positioning has disadvantages such as high power consumption, large data volume, and risk of user privacy leakage.

[0056] With the development of image sensing technology, event cameras (EVS) have emerged. Event cameras, also known as dynamic vision sensors (DVS), are a type of bionic vision sensor. Different from traditional CMOS image sensors that use frames as units, event cameras imitate the structure of the human eye and asynchronously output event information containing position, time, and polarity only when the logarithmic change of light intensity between frames exceeds a certain threshold. Compared with traditional CMOS image sensors, event cameras have the characteristics of low power consumption, small data volume, and high dynamic range.

[0057] The present application takes into account the above-mentioned characteristics of the event camera and applies the event camera to the interaction of the display device. An interactive device is designed in combination with the event camera and a supporting light source to perform positioning indication. Through the positioning mapping between the event camera and the display device, the optical center of the event camera can be projected onto the display screen of the display device, and the event camera can be moved to indicate the display screen of the display device. Since the trajectory or action of the event camera relative to the display screen varies, a variety of different control instructions can be defined in combination with a variety of different trajectories or actions, which enriches the interactive function and improves the interaction efficiency. There is no need to set up many function buttons or sensor sampling devices. At the same time, since the event camera has a small amount of data, low power consumption, and low risk of user privacy leakage, the present application can also effectively improve the data security during the interaction process.

[0058] Specifically, the present application proposes an interactive device based on an event camera, a control method based on an event camera, an electronic device and a computer-readable storage medium, which are intended to present the optical center of the event camera on the display screen of a display device, and to indicate the display screen of the display device by moving the event camera. A variety of different control instructions can also be defined in combination with a variety of different trajectories or actions of the event camera relative to the display device, so as to enrich the interactive function and improve the interactive efficiency.

[0059] In one embodiment, the interactive device based on the event camera in the present application, such as Figure 1 As shown, this may include:

[0060] a matching light source, arranged on the display device, emitting light with an intensity varying with time; and,

[0061] The main controller includes an event camera and a main control module, wherein:

[0062] An event camera, for an event camera, for moving relative to the display device, and for capturing event images of the supporting light source while moving relative to the display device;

[0063] The main control module is used to determine the position or motion trajectory of the event camera relative to the display device based on the event image, and is also used to interactively control the display device based on the position or motion trajectory.

[0064] In this embodiment, the interactive device may include a display device and a main controller. The main controller and the display device may be connected via wireless communication, such as Bluetooth connection, infrared connection, mobile network connection or WLAN connection.

[0065] The display device may include a display screen and a matching light source arranged on the display screen. In addition, the display device may include a communication module, which may be a two-way communication module or a one-way receiving module, for communicating with the main controller.

[0066] Specifically, Figure 2 As shown, the supporting light source in this embodiment may include a plurality of independent point light sources, and the plurality of independent point light sources may be fixedly arranged around the display screen. For example, four point light sources may be placed on the outside of the four corner points of the display screen to form a regular rectangle. Furthermore, two point light sources may be placed on the left side of the two corner points on the left side of the display screen, and the other two point light sources may be placed on the right side of the two corner points on the right side of the display screen, so that the height of the supporting light source is consistent with the height of the display screen and the width of the supporting light source is greater than the width of the display screen. A larger number of point light sources may also be arranged at intervals around the edge of the display screen.

[0067] It can be understood that, in theory, the number and distribution of point light sources can be flexibly set and are not restricted. For example, the point light sources are no longer arranged around the screen, but all point light sources are arranged on one side of the screen, such as the top, left or right side of the screen; and in order to effectively improve the positioning accuracy of the cursor in subsequent embodiments and reduce the difficulty of coordinate mapping, the matching light source can generally be set to a shape similar to or the same as the display screen.

[0068] It should be noted that, in the present embodiment, the light emitted by the supporting light source may be visible light, such as white light, red light, green light or blue light, and the corresponding point light source may be a laser or a light emitting diode, etc. Considering that the visible light emitted by the supporting light source may interfere with the user using the display device, the light emitted by the supporting light source may also be invisible light, such as 850nm, 940nm infrared light, and the corresponding point light source may be an infrared light emitting diode (i.e., infrared lamp beads), or invisible light in other bands such as ultraviolet light, which will not be elaborated here.

[0069] Among them, the human eye is less sensitive to light with a wavelength above 700nm, and cannot see infrared light sources of general intensity. Therefore, the use of infrared light-emitting diodes will not affect the user's screen viewing experience. In addition, considering that the EVS camera module (i.e., the event camera) only responds to changes in light intensity, therefore, in an optional embodiment of the present invention, the infrared light-emitting diodes in the matching light source can be controlled to operate in a stroboscopic light mode, for example, Figure 3 As shown, the light emitting frequency of the infrared light emitting diode is f=300 Hz.

[0070] On this basis, in more detail, Figure 2 As shown, after the supporting light source is fixedly set on the display device, the relative position of each point light source and the display screen of the display device is fixed. The relationship between each point light source and the corner point of the display screen is determined by calibration. Combined with the actual size of the display screen, the actual size of the pixel and the actual size of the point light source, the mapping of the supporting light source coordinate system to the display screen coordinate system can be realized, thereby completing the calibration of the relative position between the supporting light source and the display screen.

[0071] In this embodiment, the main controller can be an infrared remote control or a radio remote control, that is, the communication method between the corresponding communication module and the display device can be infrared communication or radio communication; the display device is configured with a communication module and a processing module, and the processing module responds to various control instructions and controls the execution of corresponding control actions.

[0072] In addition to the event camera (i.e. Figure 2 EVS camera module shown) and main control module ( Figure 2 The main controller also includes a communication module ( Figure 2 Not shown), function keys, sampling module ( Figure 2 The communication module and the main control module are arranged inside the shell.

[0073] The communication module may be a bidirectional communication module or a unidirectional transmission module, and the communication module in the main controller is communicatively connected with the communication module in the display device.

[0074] Multiple function buttons are arranged on the front side of the shell for function selection triggering. The function buttons are connected to the sampling module. The main control module is connected to the sampling module and the communication module respectively. When the function button is pressed, the sampling module will be triggered to generate a corresponding function control signal. The main control module generates a control instruction according to the function control signal, and sends the control instruction to the display device through the communication module. The display device receives and responds to the control instruction, and performs control actions such as display mode switching, channel switching, volume adjustment, and program selection.

[0075] Specifically, the function keys may include a five-way navigation key, a power key, numeric keys, a move key, a marking key, a delete key, a paste key, a capture key and the like, and the five-way navigation key may further include an up key, a down key, a "+" key, a "-" key and an "OK" key, which are not limited here.

[0076] In addition, it should be noted that the above-mentioned function keys are not limited to physical keys. In other optional embodiments of the present application, the function keys can also be virtual keys triggered based on keys or gestures. For details, please refer to the prior art and will not be repeated here.

[0077] like Figure 2 As shown, the EVS camera module (ie, event camera) in this embodiment is arranged on the shell, and the photosensitive surface of the EVS camera module is exposed from the side of the shell, which is used to collect and identify the light emitted by the matching light source.

[0078] The EVS camera module is connected to the main control module, which enables trigger control of the EVS camera module, and sends the event image collected by the EVS camera module to the main control module for calculation and processing. The main control module can calculate the position and motion trajectory of the EVS camera module relative to the display device based on the event image, and then issue control instructions based on the position and motion trajectory, map the optical center of the EVS camera module on the display device as a display cursor, and move the display cursor according to the motion trajectory of the EVS camera module relative to the display device, and interactively control the display device based on the display cursor.

[0079] In this way, when the user moves the main controller, the EVS camera module is driven to move relative to the display device, and the EVS camera module can collect multiple frames of continuous event images of the light source configured on the display device when it moves relative to the display device. Therefore, the main control module can interactively control the display device based on at least two frames of continuous event images.

[0080] Alternatively, if Figure 2 As shown, the matching light source is arranged on the frame of the display device;

[0081] On this basis, in an optional embodiment of the present application, the above-mentioned “interactively controlling the display device according to the position or motion trajectory” may include:

[0082] The optical center of the event camera is mapped onto a display device as a display cursor according to the position, the display cursor is moved according to the motion trajectory, and the display device is interactively controlled based on the display cursor.

[0083] In this embodiment, when the EVS camera module moves relative to the display device, after the main controller controls the EVS camera module at at least two positions to collect event images of the supporting light source, for each frame of the event image, the main control module in the main controller can calculate the relative position and motion trajectory of the EVS camera module relative to the supporting light source according to the light source position of the supporting light source in the event image, and then can calculate the position of the EVS camera module relative to the display device in combination with the relative position between the supporting light source and the display screen, and can map the optical center of the event camera onto the display device as a display cursor based on the above position, and perform interactive control of the display device based on the display cursor.

[0084] Specifically, for example, when the EVS camera module moves relative to the display device, for a certain frame of event image, the position of the event camera relative to the display device is determined according to the light source position of the supporting light source in the event image, and then the optical center of the event camera can be projected onto the display device as a display cursor according to the position; when the EVS camera module moves relative to the display device, for multiple frames of continuous event images, the position of the event camera relative to the display device is determined according to the light source position of the supporting light source in the event image, and the movement trajectory of the EVS camera module relative to the display device is determined according to the corresponding multiple positions of the event camera relative to the display device, and the display cursor is moved in real time according to the movement trajectory of the EVS camera module relative to the display device; in addition, when mapping or moving the display cursor, the display device can be controlled based on the display cursor, such as moving the contents of documents, pictures, etc. unfolded on the display device based on the moving display cursor, and selecting a file in a folder and copying, deleting, etc. by combining the function button and the moving display cursor, and selecting part of the contents of the unfolded document and copying, deleting, etc. by combining the function button and the moving display cursor, which will not be repeated here.

[0085] It should be noted that, in this embodiment, according to the above description, the main controller may be provided with function keys for triggering function selection, and the function keys may include a five-way navigation key, a power key, a numeric key, a movement key, a marking key, a deletion key, a paste key, a capture key, etc. On this basis, in this embodiment, the user can combine the function keys and the display cursor to interactively control the display device and perform operations such as dragging, selecting, scrolling, marking, deleting (erasing), inserting, or zooming.

[0086] Specifically, multiple function buttons are provided on the main controller. For example, the function buttons also include a control mode switching button. The control mode switching button is used to trigger a control mode selection event to interactively control the display device according to the control mode corresponding to the control mode selection event. The user can operate the function button to switch the control mode of the main controller. For example, the control mode can include a remote control mode, an indication mode and a comprehensive mode, so that the main controller enters one of the remote control mode, the indication mode and the comprehensive mode.

[0087] Among them, the remote control mode can be an ordinary button remote control mode, which can realize control actions such as display mode switching, channel switching, volume adjustment, program selection, etc.; the indication mode refers to the cursor indication mode, which can be used for positioning indication, which is particularly suitable for file content display and will not damage the eyes; however, for files, it is not enough to use the cursor to display the content. Generally, display content switching operations such as selection and page scrolling are required, as well as display content editing operations such as marking, deleting (erasing), inserting or zooming. Therefore, it is also possible to combine function keys and cursor indication to realize a comprehensive mode of multiple complex functions.

[0088] In detail, the working principle of the main controller after entering the remote control mode is as follows:

[0089] The main control module generates a control instruction based on the function control signal triggered by other function buttons except the control mode switching button, and sends it to the display device through the first communication module. The display device receives the control instruction through the second communication module, and responds to the control instruction through the internal processing module to perform control actions such as display mode switching, channel switching, volume adjustment, and program selection. It is worth noting that at this time, the EVS camera module and the supporting light source are not started.

[0090] In detail, the working principle of the main controller after entering the indication mode is as follows:

[0091] (1) The main control module sends a first enable signal and a second enable signal, the first enable signal enables the EVS camera module to start, the display device receives and responds to the second enable signal to enable the start of the supporting light source, and the supporting light source emits light with intensity that changes over time;

[0092] (2) The user moves the main controller to a suitable position, and the main controller identifies and locates the supporting light source by controlling the EVS camera module. At the same time, the main control module issues a control instruction to map the optical center of the EVS camera module onto the display device as a display cursor;

[0093] (3) Then, the mobile main controller can drive the EVS camera module to move, collect multiple frames of continuous event images of the supporting light source within a period of time through the EVS camera module, and calculate the position and motion trajectory of the EVS camera module relative to the supporting light source through the multiple frames of continuous event images of the supporting light source, and then calculate the position and motion trajectory of the EVS camera module relative to the display device in combination with the relative position relationship between the supporting light source and the display screen;

[0094] (4) Finally, the main control module issues a control instruction to move the display cursor according to the motion trajectory of the EVS camera module relative to the display screen.

[0095] In detail, the working principle of the main controller after entering the integrated mode is as follows:

[0096] (1) First, similar to the working principle of the above-mentioned indication mode, the main control module sends an enable signal to enable the EVS camera module and the supporting light source to be started, and the main controller is moved to drive the EVS camera module to move along. The EVS camera module collects the continuous position changes of the supporting light source over a period of time and calculates the position and motion trajectory of the EVS camera module relative to the display device based on the data. The main control module sends a control instruction to map the optical center of the EVS camera module on the display device as a display cursor, and moves the display cursor according to the motion trajectory of the EVS camera module relative to the display device;

[0097] (2) Secondly, more complex operations such as dragging, selecting, scrolling, marking, deleting (erasing), inserting or zooming can be achieved by combining the display cursor with the function keys.

[0098] Furthermore, in the comprehensive mode, the following operations can be performed on the document: the user holds down the movement key of the main controller while moving the display cursor to drag the document; flips the document page up by holding down the up key, and flips the document page down by holding down the down key; selects part of the text content of the document by moving the display cursor while holding down the "OK" key; enlarges or bolds the font of the selected part of the text content by holding down the "+" key; reduces the font of the selected part of the text content by holding down the "-" key; marks the background color or text color of the selected part of the text content by the mark key; deletes the content at the cursor position by the delete key; and enters content at the cursor position by the number keys.

[0099] Furthermore, in the comprehensive mode, the following operations can be performed on the picture: hold down the move key and move the display cursor to drag the picture; flip the picture up by holding down the up key, and flip the picture down by holding down the down key; hold down the "OK" key and move the display cursor to select and capture part of the picture in a rectangular box or any shape; paste the captured part of the picture by moving the display cursor in combination with the paste key; enlarge the captured part of the picture by the "+" key; reduce the captured part of the picture by the "-" key; hold down the mark key and move the display cursor to select part of the picture in a rectangular box or any shape for marking; erase the picture content by using the delete key in combination with the display cursor movement; input text or numbers by using the number keys in combination with the display cursor movement.

[0100] Furthermore, in the comprehensive mode, the following operations can be performed on the video: hold down the move key and move the display cursor to drag the video; use the up key to fast rewind the video, use the down key to fast forward the video, press and hold the "OK" key to play or pause the video, when the video is playing, select the video time period by moving the display cursor and hold down the capture key to capture the video content, when the video is paused, move the display cursor relative to the display screen and hold down the capture key to capture the current video frame content, move the display cursor and press the paste key to paste and save the captured part, press and hold the "+" key to increase the video volume, press and hold the "-" key to reduce the video volume, and use the delete key to delete the video at the cursor position.

[0101] Furthermore, in comprehensive mode, the following operations can be performed on audio: hold down the move key and move the display cursor to drag the audio; use the up key to fast rewind the audio and the down key to fast forward the audio; press and hold the "OK" key to play or pause the audio; when the audio is playing, select the audio time period by moving the display cursor and press and hold the capture key to capture the audio content; paste and save the captured part by moving the display cursor and pressing the paste key; increase the audio volume with the "+" key and decrease the audio volume with the "-" key; and delete the audio at the cursor position with the delete key.

[0102] It should be noted that in this embodiment, the control actions in the comprehensive mode are flexible and diverse, and are not limited to the above-mentioned dragging, selection, page scrolling, marking, deletion, insertion or zooming operations. The control actions can be flexibly selected and designed according to actual needs. The above are only examples of possible operations and are not limited here.

[0103] It is worth noting that in this embodiment, since the calibration parameters of the EVS camera module, the relative position of the supporting light source and the display screen are known, the position and motion trajectory of the EVS camera module relative to the display device can be analyzed and determined in combination with computer vision algorithms and related positioning algorithms based on the collected multi-frame continuous event images; and then when the display cursor is moved according to the motion trajectory of the EVS camera module relative to the display device, only the projection of the three-dimensional space motion trajectory of the EVS camera module on the display screen of the display device is considered, characterizing its up, down, left, right (XY plane where the display screen is located) motion relative to the display screen, without considering the front and back (Z-axis direction) motion, and without considering the change in the distance of the EVS camera module relative to the display device. That is to say, if the projection of the three-dimensional space motion trajectory of the EVS camera module on the display screen remains unchanged and only the distance of the EVS camera module relative to the display device changes, the corresponding display cursor will not move, but can be used for operations such as selecting or zooming the display content.

[0104] Optionally, in another optional embodiment of the present application, the above-mentioned “interactively controlling the display device according to the position or motion trajectory” may include:

[0105] Obtain operation instructions based on position or motion trajectory;

[0106] Interactively control the display device according to the operation instructions.

[0107] In this embodiment, when the EVS camera module moves relative to the display device, the main controller controls the EVS camera module to collect event images of the matching light source at at least two positions. For each frame of the event image, the main control module in the controller can calculate the position and motion trajectory of the EVS camera module relative to the display device according to the light source position of the matching light source in the event image, and can obtain operation instructions based on the position or motion trajectory, and then interactively control the display device according to the operation instructions.

[0108] It is worth noting that, according to the above-mentioned embodiments, it is necessary to combine the function keys and the display cursor to realize various complex controls of the display device in the integrated mode. The corresponding remote control operation is relatively complex, and it is necessary to press various function keys while moving the main controller. Therefore, it is necessary for this embodiment to simplify the remote control actions of various complex controls of the display device in the integrated mode, and mainly or completely rely on the action of the display cursor to remotely control to realize the complex control of the display device. That is, this embodiment can directly obtain the corresponding operation instructions to operate the display device according to the above-mentioned position or motion trajectory, and can realize the control of the display device without combining the function keys.

[0109] Specifically, for example, the working principle after the main controller enters the integrated mode may also be as follows:

[0110] (1) First, similar to the working principle of the indication mode, the main control module sends an enable signal to enable the EVS camera module and the supporting light source to be started, and the main controller is moved to drive the EVS camera module to move along. The EVS camera module collects the continuous position changes of the supporting light source over a period of time and calculates the position and motion trajectory of the EVS camera module relative to the display device based on the data. The main control module sends a control instruction to map the optical center of the EVS camera module on the display device as a display cursor, and moves the display cursor according to the motion trajectory of the EVS camera module relative to the display device.

[0111] (2) Secondly, on the basis of displaying the cursor for indication, more complex operations are achieved only through special actions of the main controller or the EVS camera module, that is, more complex operations are achieved through special actions of the display cursor, such as dragging, selecting, page scrolling, marking, deleting (erasing), inserting or zooming.

[0112] Thus, in comprehensive mode, the following complex operations can be performed on the document:

[0113] (2.1) First, move the main controller to move the display cursor to the location of the document, and then move the main controller again to shorten the distance between the EVS camera module and the display device twice while keeping the display cursor stationary, so as to select the document;

[0114] (2.2) After selecting the document in (2.1) above, move the main controller so that the display cursor moves on the display screen, and then drag the selected document;

[0115] (2.3) After selecting the document in (2.1) above, move the main controller so that the display cursor slides from left to right to the right edge of the display screen to open the document;

[0116] (2.4) After opening the document in (2.3) above, move the main controller so that the display cursor moves to the left side of the display screen and slides from bottom to top on the left side of the display screen, thereby turning up the document page;

[0117] (2.5) After opening the document in (2.3) above, the main controller is moved so that the display cursor moves to the left side of the display screen and slides from top to bottom on the left side of the display screen, thereby realizing the downward flipping of the document page;

[0118] (2.6) After opening the document in (2.3) above, the main controller is moved so that the display cursor slides downward from the upper edge of the display screen twice in succession, thereby triggering the text selection function, and then the main controller is moved again to select part of the text content of the document by the display cursor;

[0119] (2.7) After selecting part of the text content in the above (2.6), the main controller is moved to shorten the distance between the EVS camera module and the display screen while keeping the display cursor still, thereby enlarging the font of the selected part of the text content;

[0120] (2.8) After selecting part of the text content in the above (2.6), the main controller is moved to expand the distance between the EVS camera module and the display screen while keeping the display cursor still, thereby reducing the font size of the selected part of the text content;

[0121] (2.9) After selecting part of the text content in (2.6) above, move the main controller so that the display cursor moves from the right edge of the display screen to the left, thereby deleting the selected part of the text content;

[0122] (2.10) First, move the main controller to move the display cursor to the location of the document. On the basis of keeping the display cursor stationary, move the main controller again to expand the distance between the EVS camera module and the display screen twice to input the document. At this time, the display cursor becomes an electronic pen, and move the main controller to write characters on the display screen.

[0123] It is understandable that complex manipulation of files such as pictures, videos and audios can be designed similarly and will not be elaborated here; at the same time, the action of displaying the cursor and the corresponding complex manipulation can also have more optional forms, which can be flexibly designed according to actual needs and are not limited here.

[0124] In this way, this embodiment designs an interactive device based on an event camera in combination with the EVS camera module and the supporting light source, moves the main controller and the EVS camera module thereon, and keeps the display device and the supporting light source thereon stationary. At this time, the EVS camera module can collect multiple frames of continuous event images of the supporting light source within a period of time. Based on the known calibration parameters of the EVS camera module and the relative position of the supporting light source and the display screen, the position and motion trajectory of the EVS camera module relative to the display device can be determined by calculating the multiple frames of continuous event images. At this time, the optical center of the EVS camera module can be mapped on the display device as a display cursor, and the display cursor can be moved according to the motion trajectory of the EVS camera module relative to the display device. The display screen of the display device can also be manipulated according to the position and motion trajectory of the EVS camera module relative to the display device.

[0125] Among them, the calibration parameters of the EVS camera module can be pre-calibrated before leaving the factory, or can be temporarily calibrated. For example, in an optional embodiment of the present application, the EVS camera module can be calibrated in advance: the EVS camera module is calibrated using a black and white checkerboard, and the intrinsic parameter matrix, extrinsic parameter matrix, distortion coefficient and other parameters of the EVS camera module are obtained, and the parameters are saved for subsequent calls.

[0126] In general, the present application can effectively calculate the position and motion trajectory of the main controller relative to the display device through the EVS camera module on the main controller, the supporting light source on the display device and the computer vision algorithm, thereby realizing accurate sensing acquisition function. This event camera-based sensing method can be used for remote control interaction. Moreover, since the changes in the position and motion trajectory of the main controller are particularly suitable for positioning instructions or complex action control, the present application can define a variety of different control instructions in combination with a variety of different trajectories or actions, enriching the interactive function and improving the interactive efficiency. Compared with the prior art, a lot of function buttons or sensor sampling devices are no longer required, saving the interactive control cost. At the same time, since the EVS camera module has a small amount of data, low power consumption and low risk of leaking user privacy, the present application can also ensure data security during the interaction process.

[0127] In one embodiment, the present application further provides a control method based on an event camera. The method in this embodiment can be applied to the main controller in the above embodiment. The main controller includes an event camera. The main controller is connected to the display device for communication, such as Figure 4 As shown, the following steps may be included:

[0128] S10, collecting, by an event camera, an event image of a matching light source when the event camera moves relative to the display device, wherein the matching light source is arranged on the display device and emits light whose intensity varies with time;

[0129] S20, interactively controlling the display device according to the event image.

[0130] In this embodiment, the user can drive the main controller to move to manipulate the display device. In this way, when the event camera moves relative to the display device, the main controller can control the event camera to collect the position information of the supporting light source set on the display device to obtain the corresponding event image, and then the display screen can be interactively controlled according to the event image.

[0131] Optionally, in the above S20, “interactively controlling the display device according to the event image” may include:

[0132] S201, determining a position or motion trajectory of an event camera relative to a display device according to an event image;

[0133] S202, mapping the optical center of the event camera onto a display device as a display cursor according to the position, moving the display cursor according to the motion trajectory, and interactively controlling the display device based on the display cursor.

[0134] In this embodiment, after the event camera in the main controller collects multiple frames of continuous event images corresponding to the supporting light source, it can parse the event image to obtain the relative position and motion trajectory of the EVS camera module relative to the supporting light source, and then combine the relative position between the supporting light source and the display screen to calculate the position and motion trajectory of the EVS camera module relative to the display device. Based on the position, the optical center of the EVS camera module can be mapped on the display screen as a display cursor, and the display cursor can be moved according to the motion trajectory of the EVS camera module relative to the display device, and the display device can be interactively controlled based on the display cursor.

[0135] It should be noted that, in this embodiment, according to the above description, the main controller can be provided with function buttons for triggering function selection, so that the user can also operate the function buttons and move the event camera to implement more complex control operations on the display device. In this way, the main controller can interactively control the display device based on the function buttons and the display cursor. Please refer to the description of the above embodiment and will not repeat it here.

[0136] Optionally, in the above S20, “interactively controlling the display device according to the event image” may include:

[0137] S203, determining the position or motion trajectory of the event camera relative to the display device according to the event image;

[0138] S204, obtaining an operation instruction according to the position or motion trajectory;

[0139] In this embodiment, the main controller controls the event camera at at least two positions to capture event images of the supporting light sources. After calculating the position and motion trajectory of the EVS camera module relative to the display device based on the light source position of the supporting light source in the event image, the main controller can obtain operation instructions based on the position or motion trajectory, and then interactively control the display device based on the operation instructions.

[0140] It is worth noting that in this embodiment, the main controller can directly obtain the corresponding operation instructions to operate the display device according to the above position or motion trajectory, and the display device can be controlled without combining function buttons. Please refer to the description of the above embodiment and will not repeat it here.

[0141] Optionally, before the above S10, “capturing, by the event camera, an event image of the light source when the event camera moves relative to the display device”, the following may also be included:

[0142] S30, triggering a control mode selection event;

[0143] S40, responding to the control mode selection event so that the main controller enters the target control mode and performs corresponding interactive control;

[0144] Wherein, if the target control mode is the first control mode, then during interactive control, the function control signal is first triggered, and then the function control signal is responded to, a corresponding operation instruction is generated and sent to the display device to control the display device;

[0145] If the target control mode is the second control mode, then during interactive control, the matching light source is started so that the matching light source emits invisible light whose intensity varies with time, and the event camera is started to capture an event image of the matching light source when the event camera moves relative to the display device.

[0146] It should be noted that, in this embodiment, according to the above description, the function buttons of the main controller may include a control mode switching button, so that the user can operate the function button to trigger a control mode selection event to switch the control mode of the main controller. For example, the control mode may include a remote control mode, an indication mode, and a comprehensive mode, so that the main controller enters one of the remote control mode, the indication mode, and the comprehensive mode.

[0147] In this way, the main controller can respond to the control mode selection event triggered by the user operating the main controller, so that the main controller enters the target control mode and performs corresponding interactive control, wherein the target control mode can be one of the remote control mode, indication mode and comprehensive mode.

[0148] Furthermore, if it is determined that the target control mode is the first control mode (i.e., the remote control mode in this embodiment), the function control signal triggered by the user based on the main controller can be obtained, and an operation instruction corresponding to the function control signal can be generated, and the operation instruction can be sent to the display device to control the display device. For example, the audio content, video content, text content, etc. of the display device can be controlled. It is worth noting that at this time, the EVS camera module and the supporting light source are not started.

[0149] If it is determined that the target control mode is the second control mode (i.e., the indication mode or the comprehensive mode in this embodiment), the event camera in the main controller can be controlled to start during interactive control, and the matching light source on the display device can be controlled to start. The matching light source can emit invisible light whose intensity changes over time. In this way, the main controller can control the started event camera to capture the event image of the matching light source when the event camera moves relative to the display device.

[0150] In general, if Figure 5 As shown, the control method based on the event camera in this embodiment may specifically include:

[0151] Step 1, triggering the matching light source on the display device to emit light with intensity varying over time;

[0152] Step 2, move the EVS camera module, and collect multiple frames of continuous event images of the matching light source within a period of time through the EVS camera module;

[0153] Step 3: Based on multiple frames of continuous event images, calibration parameters of the EVS camera module, and the relative position of the supporting light source and the display device, a computer vision algorithm is used to calculate the position and motion trajectory of the EVS camera module relative to the display device.

[0154] Step 4: Map the optical center of the EVS camera module on the display device as a display cursor, and move the display cursor according to the motion trajectory of the EVS camera module relative to the display device;

[0155] Step 5: According to the position and movement trajectory of the EVS camera module relative to the display device, the display screen of the display device is controlled.

[0156] Among them, since the principle of the above-mentioned control method based on the event camera is similar to the principle of the above-mentioned interactive device based on the event camera, the same technical details will not be repeated. The detailed steps of the above-mentioned method can be analyzed with reference to the above-mentioned interactive device based on the event camera.

[0157] Therefore, this embodiment calculates the position and motion trajectory of the main controller relative to the display device through the EVS camera module on the main controller, the supporting light source on the display device and the computer vision algorithm, thereby realizing accurate sensing acquisition function. This event camera-based sensing method can be used for remote control interaction. Moreover, since the changes in the position and motion trajectory of the main controller are particularly suitable for positioning instructions or complex action control, this embodiment can define a variety of different control instructions in combination with a variety of different trajectories or actions, enriching the interactive function and improving the interactive efficiency. Compared with the prior art, it is no longer necessary to set many function buttons or sensor sampling devices, saving the interactive control cost. At the same time, since the EVS camera module has a small amount of data, low power consumption, and a low risk of leaking user privacy, the present application can also ensure data security during the interaction process.

[0158] Accordingly, an embodiment of the present application further provides an electronic device, such as Figure 6 As shown, Figure 6A schematic diagram of the structure of an electronic device 1100 provided in an embodiment of the present application. The electronic device 1100 includes a processor 1101 having one or more processing cores, a memory 1102 having one or more computer-readable storage media, and a computer program stored in the memory 1102 and executable on the processor. The processor 1101 is electrically connected to the memory 1102. Those skilled in the art will appreciate that the electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown, or combine certain components, or arrange components differently.

[0159] The processor 1101 is the control center of the electronic device 1100, and uses various interfaces and lines to connect various parts of the entire electronic device 1100. By running or loading software programs and / or units stored in the memory 1102, and calling data stored in the memory 1102, the processor 1101 executes various functions of the electronic device 1100 and processes data, thereby monitoring the electronic device 1100 as a whole. The processor 1101 can be a processor (Central Processing Unit, CPU), a graphics processing unit (graphics processing unit, GPU), a network processor (Network Processor, NP), etc., and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0160] In the embodiment of the present application, the processor 1101 in the electronic device 1100 will load instructions corresponding to the processes of one or more application programs into the memory 1102 according to the following steps, and the processor 1101 will run the application programs stored in the memory 1102 to implement various functions, such as:

[0161] Acquiring, by an event camera, an event image of a supporting light source when the event camera moves relative to a display device, wherein the supporting light source is arranged on the display device and emits light whose intensity varies with time;

[0162] According to the event image, the display device is interactively controlled.

[0163] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0164] Optional, such as Figure 6As shown, the electronic device 1100 further includes: a touch screen 1103, a radio frequency circuit 1104, an audio circuit 1105, an input unit 1106, and a power supply 1107. The processor 1101 is electrically connected to the touch screen 1103, the radio frequency circuit 1104, the audio circuit 1105, the input unit 1106, and the power supply 1107, respectively. Those skilled in the art can understand that Figure 6 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0165] The touch display screen 1103 can be used to display a graphical user interface and receive operation instructions generated by the user acting on the graphical user interface. The touch display screen 1103 may include a display panel and a touch panel. Among them, the display panel can be used to display information input by the user or information provided to the user and various graphical user interfaces of the electronic device, and these graphical user interfaces can be composed of graphics, text, icons, videos and any combination thereof. Optionally, the display panel can be configured in the form of a liquid crystal display (LCD, Liquid Crystal Display), an organic light-emitting diode (OLED, Organic Light-EmittingDiode) and the like. The touch panel can be used to collect the user's touch operation on or near it (such as the user uses any suitable object or attachment such as a finger, a stylus, etc. on the touch panel or near the touch panel), and generate corresponding operation instructions, and the operation instructions execute corresponding programs. Optionally, the touch panel may include two parts: a touch display system and a touch controller. Among them, the touch display system detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch display system, converts it into the touch point coordinates, and then sends it to the processor 1101, and can receive the command sent by the processor 1101 and execute it. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it is transmitted to the processor 1101 to determine the type of touch event, and then the processor 1101 provides a corresponding visual output on the display panel according to the type of touch event. In an embodiment of the present application, the touch panel and the display panel can be integrated into the touch display screen 1103 to realize the input and output functions. However, in some embodiments, the touch panel and the touch panel can be used as two independent components to realize the input and output functions. That is, the touch display screen 1103 can also be used as a part of the input unit 1106 to realize the input function.

[0166] The radio frequency circuit 1104 may be used to send and receive radio frequency signals, so as to establish wireless communication with a network device or other electronic devices through wireless communication, and to send and receive signals between the network device or other electronic devices.

[0167] The audio circuit 1105 can be used to provide an audio interface between the user and the electronic device through a speaker and a microphone. The audio circuit 1105 can transmit the electrical signal converted from the received audio data to the speaker, which is converted into a sound signal for output; on the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 1105 and converted into audio data, and then the audio data is output to the processor 1101 for processing, and then sent to another electronic device through the radio frequency circuit 1104, or the audio data is output to the memory 1102 for further processing. The audio circuit 1105 may also include an earplug jack to provide communication between an external headset and an electronic device.

[0168] The input unit 1106 may be used to receive input numbers, character information or user feature information (such as fingerprint, iris, facial information, etc.), and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0169] The power supply 1107 is used to supply power to various components of the electronic device 1100. Optionally, the power supply 1107 can be logically connected to the processor 1101 through a power management device, so that the power management device can manage charging, discharging, power consumption and other functions. The power supply 1107 can also include one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0170] although Figure 6 Not shown, the electronic device 1100 may also include a camera, a sensor, a wireless fidelity module, a Bluetooth module, etc., which will not be described in detail here.

[0171] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0172] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0173] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of computer programs are stored. The computer program can be loaded by a processor to implement any event-based camera control method provided in the embodiment of the present application. When the computer program is loaded and executed, the following steps of the event-based camera control method can be implemented:

[0174] Acquiring, by an event camera, an event image of a supporting light source when the event camera moves relative to a display device, wherein the supporting light source is arranged on the display device and emits light whose intensity varies with time;

[0175] According to the event image, the display device is interactively controlled.

[0176] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0177] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0178] Since the computer program stored in the computer-readable storage medium can execute any event-based camera control method provided in the embodiments of the present application, the beneficial effects that can be achieved by any event-based camera control method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0179] In the above-mentioned control method based on event camera, electronic device, computer-readable storage medium, and computer program product, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the above-described control method based on event camera, electronic device, computer-readable storage medium, computer program product and the specific working process and beneficial effects thereof can be referred to the description of the control method based on event camera in the above embodiment, and the specific details will not be repeated here.

[0180] The above are only preferred embodiments of the present application and do not constitute any form of limitation to the present application. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. An interactive device based on an event camera, used for interacting with a display device, characterized in that: include: a matching light source, arranged on the display device, emitting light whose intensity varies with time; as well as, The main controller includes an event camera and a main control module, wherein: The event camera is used to move relative to the display device, and is also used to capture event images of the supporting light source when moving relative to the display device; The main control module is used to determine the position or motion trajectory of the event camera relative to the display device according to the event image, and is also used to interactively control the display device according to the position or the motion trajectory.

2. The device according to claim 1, characterized in that The matching light source is arranged on the frame of the display device; The interactively controlling the display device according to the position or the motion trajectory includes: The optical center of the event camera is mapped onto the display device as a display cursor according to the position, the display cursor is moved according to the motion trajectory, and the display device is interactively controlled based on the display cursor.

3. The device according to claim 1, characterized in that The interactively controlling the display device according to the position or the motion trajectory includes: Acquiring an operation instruction according to the position or the motion trajectory; The display device is interactively controlled according to the operation instruction.

4. The device according to any one of claims 1 to 3, characterized in that: The main controller is provided with a plurality of function buttons, wherein the function buttons include a control mode switching button, and the control mode switching button is used to trigger a control mode selection event so as to interactively control the display device according to the control mode corresponding to the control mode selection event.

5. A control method based on an event camera, characterized in that: The method is applied to a main controller, the main controller includes the event camera, the main controller is communicatively connected with a display device, and the method includes: Collecting, by the event camera, an event image of a matching light source when the event camera moves relative to the display device, wherein the matching light source is arranged on the display device and emits light whose intensity varies with time; The display device is interactively controlled according to the event image.

6. The method according to claim 5, characterized in that The interactively controlling the display device according to the event image includes: Determine the position or motion trajectory of the event camera relative to the display device according to the event image; The optical center of the event camera is mapped onto the display device as a display cursor according to the position, the display cursor is moved according to the motion trajectory, and the display device is interactively controlled based on the display cursor.

7. The method according to claim 5, characterized in that The interactively controlling the display device according to the event image includes: Determine the position or motion trajectory of the event camera relative to the display device according to the event image; Acquiring an operation instruction according to the position or the motion trajectory; The display device is interactively controlled according to the operation instruction.

8. The method according to any one of claims 5 to 7, characterized in that: Before collecting, by the event camera, an event image of the supporting light source when the event camera moves relative to the display device, the method further includes: Trigger control mode selection event; Responding to the control mode selection event, so that the main controller enters the target control mode and performs corresponding interactive control; Wherein, if the target control mode is the first control mode, then during interactive control, a function control signal is first triggered, and then the function control signal is responded to, a corresponding operation instruction is generated and the operation instruction is sent to the display device to control the display device; If the target control mode is the second control mode, then during interactive control, the supporting light source is started so that the supporting light source emits invisible light whose intensity varies with time, and the event camera is started to capture an event image of the supporting light source when the event camera moves relative to the display device.

9. An electronic device, characterized in that: It includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the electronic device implements the steps of any method as claimed in claims 5 to 8.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed, the steps of the method according to any one of claims 5 to 8 are implemented.

Images