Interaction method, electronic device and system

Abstract

The present application relates to the technical field of terminals. Provided are an interaction method, an electronic device and a system. The present application can implement the positioning of a target object by means of a scanning plane formed by a light beam emitted from a first electronic device, thereby making it possible to provide a better interaction experience for a user while reducing the cost and the installation difficulty. The method comprises: a first electronic device emitting a light beam, so as to form a scanning plane at a position close to a display screen of a second electronic device, wherein the scanning plane is parallel or approximately parallel to a plane where the display screen is located; and the first electronic device acquiring a reflected light beam formed by means of the reflection of the light beam by a target object, wherein the reflected light beam is used for determining position information of an intended interaction position of the target object on the display screen, and the position information is used by the second electronic device to execute an interaction operation corresponding to the position information.

Description

Interaction methods, electronic devices and systems

[0001] This application claims priority to Chinese Patent Application No. 202411724948.1, filed with the State Intellectual Property Office of China on November 27, 2024, entitled "An Interactive Device and System", the entire contents of which are incorporated herein by reference, and to Chinese Patent Application No. 202411795284.8, filed with the State Intellectual Property Office of China on December 6, 2024, entitled "An Interactive Method, Electronic Device and System", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of terminal technology, and in particular to an interaction method, electronic device and system. Background Technology

[0003] With the development of terminal technology, electronic devices (such as smart screens and tablets) can provide touch services to users through touch technology. These touch technologies include, for example, infrared touch, infrared waveguides, ultrasonic touch, and computer-aided manufacturing (CAM) touch for capacitive displays. For instance, electronic devices can detect user actions on the display screen using their fingers or styluses, thereby providing corresponding services.

[0004] However, the implementation of touch technology requires the installation of corresponding sensing devices on electronic devices. Custom development is required for electronic devices of different sizes. Furthermore, the cost of these sensing devices is high for large screens. The detection accuracy during subsequent use is strongly correlated with the installation accuracy of the installation and maintenance personnel during the initial installation. This results in greater installation difficulty or a serious impact on the user experience due to installation and calibration issues. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides an interactive method, electronic device, and system. The technical solution provided by this application uses a scanning plane formed by a light beam emitted by a first electronic device to locate a target object, reducing costs and installation difficulty while providing users with a better interactive experience.

[0006] To achieve the above-mentioned technical objectives, this application provides the following technical solution:

[0007] In a first aspect, an interactive system is provided, comprising: a first electronic device and a second electronic device. The first electronic device is configured to: emit a light beam to form a scanning plane at a position close to the display screen of the second electronic device, the scanning plane being parallel or approximately parallel to the plane of the display screen; and acquire a reflected light beam formed by the reflection of the light beam by a target object, the reflected light beam being used to determine positional information of the target object on the display screen at the intended interactive position. The second electronic device is configured to: execute an interactive operation corresponding to the positional information.

[0008] In this way, the second electronic device can obtain the location information of the target object by simply using the scanning plane formed by the beam emitted by the first electronic device. This satisfies the user's need to interact with the second electronic device through the target object, without the need to add additional sensing devices to the second electronic device, thus reducing costs and installation difficulty.

[0009] According to the first aspect, the location information includes first location information. The second electronic device is configured to: perform a user's touch operation on the second electronic device and / or present corresponding display elements on the second electronic device based on the first location information.

[0010] Display elements include, for example, notes or animations.

[0011] For example, in response to a touch of a target object at a certain location on the display screen, the second electronic device can obtain the first location information corresponding to the touch, thereby determining the control corresponding to the first location information, triggering the control, and executing the user's touch operation. Alternatively, the second electronic device can display corresponding display elements at the location corresponding to the first location information.

[0012] According to the first aspect, or any implementation of the first aspect above, the location information includes multiple second location information. The second electronic device is used to: present a trajectory corresponding to the user's interactive operation on the second electronic device based on the multiple second location information.

[0013] For example, in response to a touch operation that moves a target object on a display screen, the second electronic device can acquire multiple associated second location information, thereby displaying the corresponding trajectory. For instance, the second electronic device can display corresponding handwriting.

[0014] In this way, the second electronic device can achieve flexible display in different usage scenarios.

[0015] According to the first aspect, or any implementation of the first aspect above, the first electronic device includes a transmitting module, a receiving module, a processing module, and an angle measurement module. The transmitting module is used to transmit a light beam. The receiving module is used to receive a reflected light beam. The processing module is used to acquire first-dimensional information based on the time of transmitting the light beam and the time of receiving the reflected light beam. The angle measurement module is used to measure second-dimensional information of the intended interaction position relative to the first electronic device. The first-dimensional information and the second-dimensional information are used to determine position information.

[0016] The angle measurement module is capable of measuring the dimensional information of the intended interaction position relative to the angle measurement module. The angle measurement module can treat this dimensional information as a second dimension of the intended interaction position relative to the first electronic device.

[0017] Optionally, the first dimension information may be, for example, time information, and the second dimension information may be, for example, angle information. The time information is used to determine the distance information of the intended interaction location relative to the first electronic device. Alternatively, the first dimension information may be, for example, distance information, and the second dimension information may be, for example, angle information.

[0018] According to the first aspect, or any implementation of the first aspect above, the first electronic device is configured to: send first-dimensional information and second-dimensional information to the second electronic device. The second electronic device is configured to: determine location information based on the first-dimensional information and the second-dimensional information.

[0019] For example, after obtaining the distance and angle information of the intended interaction location relative to itself, the first electronic device can send the distance and angle information to the second electronic device. Accordingly, the second electronic device can determine the position information of the intended interaction location on the display screen based on the received distance and angle information.

[0020] For example, after acquiring time information and the angle information of the intended interaction position relative to the first electronic device, the first electronic device can send the time information and angle information to the second electronic device. Accordingly, the second electronic device can determine the distance information of the intended interaction position relative to the first electronic device based on the received time information. Then, the second electronic device can determine the position information of the intended interaction position on the display screen based on the distance information and the angle information.

[0021] Thus, based on the location information of the first electronic device, the distance and offset angle of the intended interaction position relative to the first electronic device are determined through laser reflection information, thereby determining the location information of the intended interaction position. This simplifies the process of determining location information and eliminates the need to add complex sensing devices to the second electronic device, reducing the hardware cost of the second electronic device.

[0022] According to the first aspect, or any implementation of the first aspect above, the second electronic device is used to: filter information based on first-dimensional information and / or second-dimensional information according to information associated with the interactive area on the display screen to obtain location information.

[0023] Alternatively, the interactive area can be any of the following: full screen, application window, or area selected by the user.

[0024] Optionally, the associated information may include, for example, the size and location information of the interactive area.

[0025] In this way, the second electronic device can eliminate environmental interference, determine the interaction position between the target object and the second electronic device, and provide users with a more accurate interactive experience.

[0026] According to the first aspect, or any implementation of the first aspect above, the first electronic device is equipped with a rotation module. The rotation module is used to form a scanning plane.

[0027] Optionally, the rotation module may include, for example, a rotating mirror. Exemplarily, during the high-frequency emission of a light beam by the first electronic device, the rotating mirror rotates the beam, thereby forming a fan-shaped scanning plane on the display surface of the second electronic device.

[0028] Optionally, the rotation module may include, for example, a motor for rotating the transmitting module, such that as the transmitting module rotates, the beam emitted by the transmitting module forms a scanning plane.

[0029] In this way, a scanning plane is formed through mechanical scanning, thereby enabling the positioning of the target object.

[0030] According to the first aspect, or any implementation thereof, the first electronic device is configured with a transmitting module and a receiving module, wherein the transmitting module includes multiple transmitting units and / or the receiving module includes multiple receiving units. The multiple transmitting units are used to transmit a light beam forming a scanning plane. The multiple receiving units are used to receive reflected light beams.

[0031] In this way, a scanning plane is formed through multi-input multi-output electrical scanning, thereby enabling the positioning of the target object.

[0032] According to the first aspect, or any implementation of the first aspect above, the target object is an active device, and the target object includes a pressure sensor. The target object is used to: transmit pressure data detected by the pressure sensor to a second electronic device. The second electronic device is further used to: perform an interactive operation corresponding to the location information based on the received pressure data.

[0033] In this way, by combining pressure data, the system can provide users with a realistic writing experience on the display screen using the target object, thus avoiding problems such as ink leakage and scribbling.

[0034] According to the first aspect, or any implementation of the first aspect above, the second electronic device is further used to: adjust the presentation effect of the display element set at the first position information according to the pressure data.

[0035] According to the first aspect, or any of the above implementations of the first aspect, the presentation effect includes the thickness of the presentation trajectory or the animation of the displayed elements.

[0036] In this way, the second electronic device can also adjust the thickness of the handwriting according to the changes in the received pressure data, thereby providing users with a better interactive experience.

[0037] According to the first aspect, or any implementation of the first aspect above, the target object is an active device, the target object includes an attitude sensor, the attitude sensor is used to acquire the attitude data of the target object, the attitude data is used to predict the handwriting position, and / or, acquire correction information to determine the position information.

[0038] According to the first aspect, or any implementation of the first aspect above, the second electronic device is further configured to: receive attitude data; and obtain correction information based on the attitude data to determine position information.

[0039] Optionally, the attitude sensor can be, for example, an inertial measurement unit (IMU). For instance, an IMU can help a target determine its motion state and attitude by measuring dynamic information such as acceleration and angular velocity.

[0040] In this way, by combining posture data, the second electronic device can provide users with a more realistic touch experience and avoid positional deviations that could affect user experience.

[0041] According to the first aspect, or any of the above implementations of the first aspect, the target object includes multiple target objects.

[0042] According to the first aspect, or any implementation of the first aspect above, the location information includes location information of multiple target objects at multiple intentional interaction locations on the display screen. The second electronic device is configured to: execute multiple interactive operations corresponding to the location information of the multiple intentional interaction locations.

[0043] For example, multiple users each hold a stylus and perform touch operations on the display screen. The first electronic device can locate multiple styluses, and the second electronic device needs to respond to multiple interactive operations from these styluses. Each stylus is a target object.

[0044] For example, a user might use multiple fingers to perform multi-finger operations on a display screen. In this case, the first electronic device can locate the multiple fingers, and the second electronic device needs to respond to the multiple interactive operations of those fingers, which can be combined into a single finger operation. Each finger is a target object.

[0045] The process for determining the location information of each target object can refer to the process for determining the location information of the target objects described above.

[0046] In this way, multi-touch functionality of the second electronic device is achieved through the detection of multiple target objects.

[0047] According to the first aspect, or any implementation of the first aspect above, the first electronic device is independently installed on the second electronic device, or the first electronic device is integrated into the second electronic device.

[0048] For example, the first electronic device can be integrated with the camera or microphone modules of the second electronic device into a single module.

[0049] According to the first aspect, or any implementation of the first aspect above, the first electronic device is located in the middle of the upper frame of the second electronic device.

[0050] The middle position includes the exact middle position or the position near the exact middle position.

[0051] According to the first aspect, or any implementation of the first aspect above, the target object is an active device or a passive object. Active devices include styluses or touch gloves, and passive objects include fingers. And / or, the first electronic device is any one of a direct time-of-flight (DTOF) radar, an indirect time-of-flight (ITOF) radar, a binocular radar, a structured light radar, or an electronically scanned solid-state lidar. And / or, the second electronic device is a smart screen.

[0052] Among them, the smart screen can be a screen for home or office use.

[0053] According to the first aspect, or any implementation of the first aspect above, the light beam is laser or infrared light.

[0054] Secondly, an interaction method is provided, applied to a first electronic device. The method includes: emitting a light beam to form a scanning plane at a position close to the display screen of a second electronic device, the scanning plane being parallel or approximately parallel to the plane of the display screen; acquiring a reflected light beam formed by the reflection of the light beam by a target object, the reflected light beam being used to determine position information of the target object at an intended interaction position on the display screen, and the position information being used by the second electronic device to execute an interaction operation corresponding to the position information.

[0055] According to the second aspect, the first electronic device includes a transmitting module, a receiving module, a processing module, and an angle measurement module. Transmitting a light beam includes: transmitting the light beam through the transmitting module. Acquiring a reflected light beam formed by the reflection of the light beam by a target object includes: receiving the reflected light beam through the receiving module. The method further includes: acquiring first-dimensional information based on the time of transmitting the light beam and the time of receiving the reflected light beam through the processing module. Measuring second-dimensional information of the intended interaction position relative to the first electronic device through the angle measurement module; wherein the first-dimensional information and the second-dimensional information are used to determine position information.

[0056] According to the second aspect, or any implementation of the second aspect above, the method further includes: sending first dimension information and second dimension information to the second electronic device to instruct the second electronic device to determine location information based on the first dimension information and the second dimension information.

[0057] According to the second aspect, or any implementation of the second aspect above, the first electronic device is equipped with a rotation module; the method further includes: forming a scanning plane by means of the rotation module.

[0058] According to the second aspect, or any implementation thereof, the first electronic device is configured with a transmitting module and a receiving module, wherein the transmitting module includes multiple transmitting units and / or the receiving module includes multiple receiving units. Transmitting a light beam includes: transmitting a light beam forming a scanning plane through the multiple transmitting units. Acquiring a reflected light beam formed by the reflection of the light beam by a target object includes: receiving the reflected light beam through the multiple receiving units.

[0059] According to the second aspect, or any implementation of the second aspect above, the first electronic device is independently installed on the second electronic device, or the first electronic device is integrated into the second electronic device.

[0060] According to the second aspect, or any implementation of the second aspect above, the first electronic device is located in the middle of the upper frame of the second electronic device.

[0061] According to the second aspect, or any implementation of the second aspect above, the target object is an active device or a passive object. Active devices include styluses or touch gloves, and passive objects include fingers. And / or, the first electronic device is any one of a direct time-of-flight (DTOF) radar, an indirect time-of-flight (ITOF) radar, a binocular radar, a structured light radar, or an electronically scanned solid-state lidar. And / or, the second electronic device is a smart screen.

[0062] According to the second aspect, or any implementation of the second aspect above, the beam is a laser beam or an infrared beam.

[0063] Thirdly, an interaction method is provided, applied to a second electronic device. The method includes: acquiring position information of the intended interaction location of a target object on the display screen of the second electronic device; wherein the position information is determined by a reflected beam formed by a light beam emitted by a first electronic device and the reflection of the light beam by the target object, the light beam forming a scanning plane near the display screen, the scanning plane being parallel or approximately parallel to the plane of the display screen. Based on the position information, an interaction operation corresponding to the position information is executed.

[0064] According to the third aspect, the location information includes first location information. Based on the location information, performing interactive operations corresponding to the location information includes: based on the first location information, performing a user's touch operation on the second electronic device and / or presenting corresponding display elements on the second electronic device.

[0065] According to the third aspect, or any implementation of the third aspect above, the location information includes multiple second location information; according to the location information, the corresponding interactive operation is executed, including: according to the multiple second location information, presenting the trajectory corresponding to the user's interactive operation on the second electronic device.

[0066] According to the third aspect, or any implementation thereof, obtaining the position information of the intended interaction location of the target object on the display screen of the second electronic device includes: receiving first-dimensional information and second-dimensional information sent by the first electronic device; wherein the first-dimensional information is determined based on the emission time of the light beam and the reception time of the reflected light beam, and the second-dimensional information is the angle information of the intended interaction location relative to the first electronic device. The position information is determined based on the first-dimensional information and the second-dimensional information.

[0067] According to the third aspect, or any implementation of the third aspect above, the location information is determined based on the first dimension information and the second dimension information, including: based on the information associated with the interactive area on the display screen, information is filtered based on the first dimension information and / or the second dimension information to obtain the location information.

[0068] According to the third aspect, or any implementation of the third aspect above, the target object is an active device, including a pressure sensor; based on the location information, the corresponding interactive operation is executed, including: receiving pressure data detected by the pressure sensor sent by the target object; and executing the corresponding interactive operation based on the pressure data.

[0069] According to the third aspect, or any of the above implementations of the third aspect, based on the pressure data, perform the interactive operation corresponding to the position information, including: based on the pressure data, adjusting the presentation effect of the display element of the second electronic device at the first position information.

[0070] According to the third aspect, or any of the above-mentioned third aspects, the presentation effect includes the thickness of the presentation trajectory or the animation effect of the displayed elements.

[0071] According to the third aspect, or any implementation of the third aspect above, the target object is an active device, the target object includes an attitude sensor, the attitude sensor is used to acquire the attitude data of the target object, the attitude data is used to predict the handwriting position, and / or, acquire correction information to determine the position information.

[0072] According to the third aspect, or any implementation of the third aspect above, obtaining the position information of the intended interactive position of the target object on the display screen of the second electronic device includes: receiving attitude data; and obtaining correction information based on the attitude data to determine the position information.

[0073] According to the third aspect, or any of the above-mentioned third aspects, the target object includes multiple target objects.

[0074] According to the third aspect, or any implementation of the third aspect above, the location information includes the location information of multiple target objects at multiple intentional interaction positions on the display screen; according to the location information, the corresponding interactive operation is executed, including: according to the location information of the multiple intentional interaction positions, the multiple interactive operations corresponding to the location information of the multiple intentional interaction positions are executed.

[0075] According to the third aspect, or any of the above implementations of the third aspect, the first electronic device is independently installed on the second electronic device, or the first electronic device is integrated into the second electronic device.

[0076] According to the third aspect, or any implementation of the third aspect above, the first electronic device is located in the middle of the upper frame of the second electronic device.

[0077] According to the third aspect, or any implementation of the third aspect above, the target object is an active device or a passive object. Active devices include styluses or touch gloves, and passive objects include fingers. And / or, the first electronic device is any one of a direct time-of-flight (DTOF) radar, an indirect time-of-flight (ITOF) radar, a binocular radar, a structured light radar, or an electronically scanned solid-state lidar. And / or, the second electronic device is a smart screen.

[0078] According to the third aspect, or any of the above-mentioned third aspects, the beam is a laser beam or an infrared beam.

[0079] Fourthly, a first electronic device is provided. The first electronic device includes a processor and a memory coupled to the processor. The memory stores computer program code, which includes computer instructions. When the processor reads the computer instructions from the memory, the first electronic device performs the following actions: emitting a light beam to form a scanning plane at a position close to a display screen of a second electronic device, the scanning plane being parallel or approximately parallel to the plane of the display screen; and acquiring a reflected light beam formed by the reflection of the light beam from a target object, the reflected light beam being used to determine position information of the target object at an intended interactive position on the display screen, the position information being used by the second electronic device to perform an interactive operation corresponding to the position information.

[0080] According to the fourth aspect, the first electronic device further includes a transmitting module, a receiving module, a processing module, and an angle measurement module. Transmitting a light beam includes: transmitting the light beam through the transmitting module. Acquiring a reflected light beam formed by the reflection of the light beam by a target object includes: receiving the reflected light beam through the receiving module. When the processor reads computer instructions from memory, it also causes the first electronic device to execute: acquiring first-dimensional information based on the time of transmitting the light beam and the time of receiving the reflected light beam through the processing module. Measuring second-dimensional information of the intended interaction position relative to the first electronic device through the angle measurement module; wherein the first-dimensional information and the second-dimensional information are used to determine position information.

[0081] According to the fourth aspect, or any implementation of the fourth aspect above, when the processor reads computer instructions from memory, it also causes the first electronic device to execute: sending first dimension information and second dimension information to the second electronic device to instruct the second electronic device to determine the location information based on the first dimension information and the second dimension information.

[0082] According to the fourth aspect, or any implementation of the fourth aspect above, the first electronic device is equipped with a rotation module; when the processor reads computer instructions from the memory, it also causes the first electronic device to perform: forming a scanning plane by means of the rotation module.

[0083] According to the fourth aspect, or any implementation of the fourth aspect above, the first electronic device is configured with a transmitting module and a receiving module, wherein the transmitting module includes multiple transmitting units and / or the receiving module includes multiple receiving units. Transmitting a light beam includes: transmitting a light beam forming a scanning plane through the multiple transmitting units. Acquiring a reflected light beam formed by the reflection of the light beam by a target object includes: receiving the reflected light beam through the multiple receiving units.

[0084] According to the fourth aspect, or any of the above implementations of the fourth aspect, the first electronic device is independently installed on the second electronic device, or the first electronic device is integrated into the second electronic device.

[0085] According to the fourth aspect, or any implementation of the fourth aspect above, the first electronic device is located in the middle of the upper frame of the second electronic device.

[0086] According to the fourth aspect, or any implementation of the fourth aspect above, the target object is an active device or a passive object. Active devices include styluses or touch gloves, and passive objects include fingers. And / or, the first electronic device is any one of a direct time-of-flight (DTOF) radar, an indirect time-of-flight (ITOF) radar, a binocular radar, a structured light radar, or an electronically scanned solid-state lidar. And / or, the second electronic device is a smart screen.

[0087] According to the fourth aspect, or any of the above implementations of the fourth aspect, the beam is a laser beam or an infrared beam.

[0088] Fifthly, a second electronic device is provided. The second electronic device includes a processor, a memory, and a display screen, the memory and display screen being coupled to the processor. The memory stores computer program code, which includes computer instructions. When the processor reads the computer instructions from the memory, the second electronic device executes: acquiring position information of the intended interactive position of a target object on the display screen of the second electronic device; wherein the position information is determined by a reflected beam formed by a light beam emitted by a first electronic device and the reflection of the light beam by the target object, the light beam forming a scanning plane near the display screen, the scanning plane being parallel or approximately parallel to the plane of the display screen. Based on the position information, the corresponding interactive operation is executed.

[0089] According to the fifth aspect, the location information includes first location information. Based on the location information, performing interactive operations corresponding to the location information includes: based on the first location information, performing a user's touch operation on the second electronic device and / or presenting corresponding display elements on the second electronic device.

[0090] According to the fifth aspect, or any implementation of the fifth aspect above, the location information includes multiple second location information; according to the location information, the corresponding interactive operation is executed, including: according to the multiple second location information, presenting the trajectory corresponding to the interactive operation with the user on the second electronic device.

[0091] According to the fifth aspect, or any implementation thereof, obtaining the position information of the intended interaction position of the target object on the display screen of the second electronic device includes: receiving first-dimensional information and second-dimensional information sent by the first electronic device; wherein the first-dimensional information is determined based on the emission time of the light beam and the reception time of the reflected light beam, and the second-dimensional information is the angle information of the intended interaction position relative to the first electronic device. The position information is determined based on the first-dimensional information and the second-dimensional information.

[0092] According to the fifth aspect, or any implementation of the fifth aspect above, the location information is determined based on the first dimension information and the second dimension information, including: based on the information associated with the interactive area on the display screen, information is filtered based on the first dimension information and / or the second dimension information to obtain the location information.

[0093] According to the fifth aspect, or any implementation of the fifth aspect above, the target object is an active device, including a pressure sensor; based on the location information, the corresponding interactive operation is executed, including: receiving pressure data detected by the pressure sensor sent by the target object; and executing the corresponding interactive operation based on the pressure data.

[0094] According to the fifth aspect, or any of the above implementations of the fifth aspect, based on the pressure data, perform the interactive operation corresponding to the position information, including: based on the pressure data, adjusting the presentation effect of the display element of the second electronic device at the first position information.

[0095] According to the fifth aspect, or any of the implementation methods of the fifth aspect above, the presentation effect includes the thickness of the presentation trajectory or the animation effect of the displayed elements.

[0096] According to the fifth aspect, or any implementation of the fifth aspect above, the target object is an active device, the target object includes an attitude sensor, the attitude sensor is used to acquire the attitude data of the target object, the attitude data is used to predict the handwriting position, and / or, acquire correction information to determine the position information.

[0097] According to the fifth aspect, or any implementation of the fifth aspect above, obtaining the position information of the intended interactive position of the target object on the display screen of the second electronic device includes: receiving attitude data; and obtaining correction information based on the attitude data to determine the position information.

[0098] According to the fifth aspect, or any of the implementations of the fifth aspect above, the target object includes multiple target objects.

[0099] According to the fifth aspect, or any implementation of the fifth aspect above, the location information includes the location information of multiple target objects at multiple intentional interaction positions on the display screen; according to the location information, the corresponding interactive operation is executed, including: according to the location information of the multiple intentional interaction positions, the multiple interactive operations corresponding to the location information of the multiple intentional interaction positions are executed.

[0100] According to the fifth aspect, or any implementation of the fifth aspect above, the first electronic device is independently installed on the second electronic device, or the first electronic device is integrated into the second electronic device.

[0101] According to the fifth aspect, or any implementation of the fifth aspect above, the first electronic device is located in the middle of the upper frame of the second electronic device.

[0102] According to the fifth aspect, or any implementation of the fifth aspect above, the target object is an active device or a passive object. Active devices include styluses or touch gloves, and passive objects include fingers. And / or, the first electronic device is any one of a direct time-of-flight (DTOF) radar, an indirect time-of-flight (ITOF) radar, a binocular radar, a structured light radar, or an electronically scanned solid-state lidar. And / or, the second electronic device is a smart screen.

[0103] According to the fifth aspect, or any of the implementations of the fifth aspect above, the beam is a laser beam or an infrared beam.

[0104] Sixthly, an electronic device is provided, which has the function of implementing the interaction method as described in the second aspect and any of its possible implementations; or, the electronic device has the function of implementing the interaction method as described in the third aspect and any of its possible implementations. This function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described function.

[0105] A seventh aspect provides a computer-readable storage medium. The computer-readable storage medium stores a computer program (also referred to as instructions or code) that, when executed by an electronic device, causes the electronic device to perform the method of the second aspect or any embodiment of the second aspect; or causes the electronic device to perform the method of the third aspect or any embodiment of the third aspect.

[0106] Eighthly, a computer program product is provided that, when run on an electronic device, causes the electronic device to perform the method of the second aspect or any one of the embodiments of the second aspect; or causes the electronic device to perform the method of the third aspect or any one of the embodiments of the third aspect.

[0107] A ninth aspect provides a circuit system including a processing circuit configured to perform the method of the second aspect or any embodiment of the second aspect; or, the processing circuit is configured to perform the method of the third aspect or any embodiment of the third aspect.

[0108] In a tenth aspect, a chip system is provided, including at least one processor and at least one interface circuit, wherein the at least one interface circuit is configured to perform transceiver functions and send instructions to the at least one processor, wherein when the at least one processor executes the instructions, the at least one processor performs the method of the second aspect or any embodiment of the second aspect; or, the at least one processor performs the method of the third aspect or any embodiment of the third aspect.

[0109] The technical effects of the aforementioned aspects can be referenced from each other, and will not be elaborated further here. Attached Figure Description

[0110] Figure 1 is a schematic diagram of a touch position determination scenario provided in an embodiment of this application;

[0111] Figure 2 is a schematic diagram of a touch position determination scenario provided in an embodiment of this application;

[0112] Figure 3 is a schematic diagram of a communication system in which the interaction method provided in the embodiments of this application is applied;

[0113] Figure 4A is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application;

[0114] Figure 4B is a schematic diagram of the hardware structure of the first electronic device 100 provided in an embodiment of this application;

[0115] Figure 4C is a schematic diagram of the hardware structure of the second electronic device 200 provided in an embodiment of this application;

[0116] Figure 4D is a schematic diagram of the hardware structure of the target object 300 provided in the embodiment of this application;

[0117] Figure 5 is a schematic diagram of the positioning device provided in an embodiment of this application;

[0118] Figure 6 is a schematic diagram of module interaction provided in an embodiment of this application;

[0119] Figure 7 is a schematic diagram of a positioning device location determination scenario provided in an embodiment of this application;

[0120] Figure 8 is a schematic diagram of the laser scanning plane provided in an embodiment of this application;

[0121] Figure 9 is a schematic diagram of a point cloud data processing scenario provided in an embodiment of this application;

[0122] Figure 10 is a schematic diagram of a pressure data detection scenario provided in an embodiment of this application;

[0123] Figure 11 is a schematic diagram of a pressure data detection scenario provided in an embodiment of this application;

[0124] Figure 12 is a schematic diagram of an attitude data detection scenario provided in an embodiment of this application;

[0125] Figure 13 is a schematic diagram of the attitude data detection scenario provided in the embodiment of this application;

[0126] Figure 14 is a schematic diagram of an electronically scanned solid-state lidar scenario provided in an embodiment of this application;

[0127] Figure 15 is a schematic diagram of the interaction method provided in an embodiment of this application;

[0128] Figure 16 is a schematic diagram of the structure of the first electronic device provided in an embodiment of this application;

[0129] Figure 17 is a schematic diagram of the structure of the second electronic device provided in an embodiment of this application. Detailed Implementation

[0130] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one or more (including two).

[0131] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0132] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0133] In some embodiments, touch technologies include, for example, infrared touch, ultrasonic touch, infrared waveguides, capacitive displays, and external CAM touch. Electronic devices can use touch technology to detect user actions on the display screen using their fingers or styluses, thereby providing corresponding services. For example, an electronic device can display corresponding handwriting on the display screen based on the user's touch operation, thus achieving image drawing. Another example is that an electronic device can launch a corresponding application based on the user's touch operation on the displayed application icon.

[0134] For example, in infrared touch technology, as shown in Figure 1(a), an infrared matrix is ​​installed around the display screen of an electronic device, capable of emitting or receiving infrared signals. When a user touches the display screen with their finger or the screen itself, it partially blocks the emission and reception of infrared signals. Thus, the electronic device can determine the touch location on the display screen by judging the change in the infrared signal's optical path and respond accordingly, thereby providing touch services.

[0135] For example, in infrared waveguide touch technology, as shown in Figure 1(b), the propagation characteristics of infrared light in specific materials are utilized to guide the infrared beam to the surface of the display screen through a waveguide. When a finger or stylus touches the surface of the display screen, the transmission path and efficiency of the optical waveguide are changed, thereby realizing the sensing and positioning of touch events.

[0136] For example, in capacitive display technology, electronic devices are equipped with capacitive displays, which include multiple capacitor modules. When a conductor such as a finger or stylus approaches the display, the capacitance value of the capacitor modules changes. The electronic device can then determine a touch event based on this change in capacitance, thereby providing touch services.

[0137] For example, in external CAM touch technology, as shown in Figure 2(a), a camera is installed above the electronic device, and infrared light is used to illuminate a light screen parallel to the display screen. During user operation, when the stylus touches the light screen, as shown in Figure 2(b), the camera captures an image to obtain the touch position of the stylus on the light screen. In this way, touch services are provided by recognizing the touch position.

[0138] It can be seen that the implementation of touch technology requires the installation of corresponding sensing devices on electronic devices, which are costly and difficult to install.

[0139] For example, as shown in Figure 1(a), the installation of sensors may also lead to an increase in the width of the electronic device's bezel, affecting the product's appearance and user experience.

[0140] For example, as shown in Figure 2(a), it is difficult to ensure that the light curtain is parallel to the display screen and as close to the display screen as possible when manually installed. This places high demands on the installation by the pre-installation and maintenance personnel; otherwise, it will directly affect the detection accuracy and thus the user experience. Furthermore, due to the limitation of the camera's field of view (FOV), the long arm of the camera in this external CAM needs to be long enough to capture the entire light curtain corresponding to the display screen, affecting the product's appearance. In addition, the hardware cost of this external CAM is relatively high.

[0141] Therefore, this application provides an interactive method that, based solely on miniaturized accessories, can provide users with a better interactive experience without affecting their viewing experience.

[0142] Figure 3 is a schematic diagram of a communication system for the interaction method provided in this application embodiment. As shown in Figure 3, the communication system includes a first electronic device 100, a second electronic device 200, and a target object 300.

[0143] Optionally, the second electronic device 200 may be a smart screen, tablet computer, mobile phone, personal computer (PC), projection device, ultra-mobile personal computer (UMPC), netbook, artificial intelligence (AI) device, or other terminal device with a display screen. The operating system installed on the second electronic device 200 may include, but is not limited to, […]. Alternatively, other operating systems may be used. This application does not limit the specific type of the second electronic device 200 or the operating system installed on it.

[0144] Optionally, the first electronic device 100 may be, for example, an accessory device to the second electronic device 200. Optionally, the first electronic device 100 may be installed independently on the second electronic device 200, or it may be directly configured within the second electronic device 200, for example, integrated with an existing module in the second electronic device 200. In this case, the first electronic device 100 may also be referred to as a first electronic device or a first electronic module, etc. Optionally, the first electronic device 100 may be, for example, a positioning device or positioning apparatus, used to locate the touch position of the target object 300 on the display screen of the second electronic device 200.

[0145] Optionally, the target object 300 can be an active device or a passive device. For example, the target object 300 can be a stylus, a touch glove, or other terminal device capable of touch operation on the display screen of the second electronic device 200. Alternatively, the target object 300 can be a whiteboard marker, a long cylindrical object, or a user's finger. This application does not limit the specific type of the target object 300.

[0146] In some embodiments, when the target device 300 is an active device, a wireless communication connection is established between the target device 300 and the second electronic device 200. The wireless communication technology used to establish this connection includes, but is not limited to, at least one of the following: Bluetooth (BT) (e.g., classic Bluetooth or Bluetooth Low Energy (BLE)), SparkLink Low Energy (SLE), wireless local area networks (WLAN) (such as Wi-Fi), near field communication (NFC), Zigbee, frequency modulation (FM), infrared (IR), etc. In some possible designs, a wireless communication link may also be established between the target device 300 and the first electronic device 100.

[0147] In some embodiments, a communication connection is established between the first electronic device 100 and the second electronic device 200. Optionally, this communication connection can be a wired communication connection or a wireless communication connection based on the aforementioned wireless communication technology. For example, a wired communication connection can be established between the first electronic device 100 and the second electronic device 200 based on an interface such as a universal serial bus (USB) interface.

[0148] Optionally, the first electronic device 100, the second electronic device 200, and the active target 300 in the embodiments of this application can be implemented by different devices. Different devices can have the same, similar, or different hardware structures, such as the hardware structure shown in Figure 4A.

[0149] For example, taking the hardware structure of the electronic device shown in Figure 4A as an example, the hardware structure of the first electronic device 100, the second electronic device 200, and the active target 300 will be described.

[0150] As shown in Figure 4A, the electronic device may include a processor 401, a memory 402, and a communication interface 403, etc.

[0151] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0152] Processor 401 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. The different processing units may be independent devices or integrated into one or more processors.

[0153] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0154] The processor 401 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 401 is a cache memory. This memory can store instructions or data that the processor 401 has just used or that are used repeatedly. If the processor 401 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 401, and thus improves the efficiency of the system.

[0155] In a specific implementation, as one example, processor 401 may include one or more CPUs, such as CPU0 and CPU1 in FIG4A.

[0156] In a specific implementation, as one embodiment, the electronic device may include multiple processors, such as processor 401 and processor 4011 in FIG. 4A. Each of these processors may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0157] The memory (402) can be random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions. It can also be electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disk storage, magnetic disk storage media, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory can exist independently and be connected to the processor via communication lines. The memory can also be integrated with the processor.

[0158] The memory 402 stores computer execution instructions for implementing the solutions of this application, and its execution is controlled by the processor 401. The processor 401 executes the computer execution instructions stored in the memory 402, thereby implementing the data processing method provided in the following embodiments of this application.

[0159] Optionally, the computer execution instructions in the embodiments of this application may also be referred to as application code, instructions, computer program or other names, and the embodiments of this application do not specifically limit them.

[0160] Communication interface 403 is used for communication with other devices. In this embodiment, the communication interface can be a module, circuit, bus, interface, transceiver, or other device capable of communication functions, used for communication with other devices. Optionally, when the communication interface is a transceiver, the transceiver can be a separately configured transmitter used to send information to other devices, or it can be a separately configured receiver used to receive information from other devices. The transceiver can also be a component that integrates sending and receiving information functions; this embodiment does not limit the specific implementation of the transceiver.

[0161] In some examples, communication interface 403 can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), Starlink, and near field communication (NFC) technologies. Communication interface 403 can be one or more devices integrating at least one communication processing module. Communication interface 403 receives electromagnetic waves via antenna 2, frequency-modulates and filters the electromagnetic wave signals, and sends the processed signal to processor 401. Communication interface 403 can also receive signals to be transmitted from processor 401, frequency-modulate and amplify them, and then convert them into electromagnetic waves for radiation via antenna 2.

[0162] In some examples, the wireless communication function of an electronic device can be implemented through an antenna, a communication interface 403, a modem processor, and a baseband processor.

[0163] Based on the introduction of the hardware structure of the electronic device shown in Figure 4A above, the hardware structures of the first electronic device 100, the second electronic device 200, and the active target 300 will be introduced below to help understand the differences in the hardware structures of the first electronic device 100, the second electronic device 200, and the active target 300.

[0164] Figure 4B is a schematic diagram of the hardware structure of another first electronic device 100 provided in an embodiment of this application.

[0165] As shown in Figure 4B, the first electronic device 100 includes the processor 401, memory 402, and communication interface 403 mentioned above, as well as modules such as a transmitting module 404, a receiving module 405, a mirror group 406, a motor 407, and an angle measurement module 408.

[0166] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the first electronic device 100. In other embodiments of this application, the first electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0167] The emitting module 404 is used to emit a light beam. Optionally, the light beam is, for example, a laser beam or an infrared beam. Optionally, the emitting module 404 emits the light beam to form a scanning plane at a position close to the display screen of the second electronic device 200, the scanning plane being parallel or approximately parallel to the plane of the display screen of the second electronic device 200. Optionally, when a user performs a touch operation on the display screen of the second electronic device 200 using a target object 300, the light beam forming the scanning plane can illuminate the target object 300. Then, the second electronic device 200 can use this light beam to locate the operating position of the target object 300 on the display screen of the second electronic device 200. Specific positioning methods are detailed in the relevant embodiments below and will not be repeated here.

[0168] The receiving module 405 is used to receive the reflected beam formed by the reflection of the beam emitted by the transmitting module by the target object.

[0169] The transmitting module 404 and the receiving module 405 can be integrated into a single module, such as a transceiver module; or they can be two separate modules. Optionally, the transceiver module may include multiple receiving units and / or multiple transmitting units, which can be implemented by a transceiver or transceiver-related circuit components.

[0170] The mirror assembly 406 is used to change the angle of the light beam emitted by the transmitting module 404, thereby achieving beam rotation and forming a scanning plane at a position close to the display screen of the second electronic device 200. Optionally, the mirror assembly 406 includes multiple mirrors, and the mirror assembly 406 can also be described as a rotating mirror.

[0171] Motor 407 is used to drive mirror group 406 or emission module 404 to rotate, thereby realizing the rotation of the beam and forming a scanning plane at a position close to the display screen of the second electronic device 200.

[0172] Angle measurement module 408 is used to measure the angle information of the intended interactive position of the target object 300 on the display screen of the second electronic device 200 relative to the first electronic device 100.

[0173] The processor 401 can also be used to measure the distance information of the intended interactive position of the target object 300 on the display screen of the second electronic device 200 relative to the first electronic device 100. For example, the processor 401 can acquire the time when the transmitting module 404 emits a light beam and the time when the receiving module 405 receives the reflected light beam, thus acquiring time information. The processor 401 can then determine the distance information based on this time information.

[0174] Figure 4C is a schematic diagram of the hardware structure of another second electronic device 200 provided in an embodiment of this application.

[0175] As shown in Figure 4C, the second electronic device 200 includes the processor 401, memory 402, and communication interface 403 mentioned above, as well as a display screen 409, camera 410, universal serial bus (USB) interface 411, high-definition multimedia interface (HDMI) 412, audio module 413, power supply module 414, and buttons 415.

[0176] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the second electronic device 200. In other embodiments of this application, the second electronic device 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0177] Display screen 409 is used to display images, videos, etc. Display screen 409 includes a display panel. The display panel can be manufactured using a liquid crystal display (LCD), such as an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini-LED, a micro-LED, a micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include one or N displays 409, where N is a positive integer greater than 1.

[0178] The second electronic device 200 implements display functions through a GPU, a display screen 409, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 409 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. The processor 401 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0179] In some examples, the second electronic device 200 can implement corresponding touch functions through the display screen 409.

[0180] Camera 410 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the second electronic device 200 may include one or N cameras 410, where N is a positive integer greater than 1.

[0181] Alternatively, the second electronic device 200 may not be equipped with a camera 410.

[0182] USB interface 411 is an interface compliant with the USB standard specification, specifically a Mini USB interface, Micro USB interface, USB Type-C interface, etc. USB interface 411 can be used for data transfer between the second electronic device 200 and peripheral devices. For example, the second electronic device 200 can establish a wired communication connection with the first electronic device 100 through USB interface 411. Alternatively, the second electronic device 200 can also establish a wireless communication connection with the first electronic device 100 through communication interface 403.

[0183] HDMI 412 can be used to transmit high-quality video signals and multi-channel audio signals, avoiding the quality loss of traditional analog signal transmission.

[0184] The audio module 413 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 413 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 413 may be located in the processor 401, or some functional modules of the audio module 413 may be located in the processor 401.

[0185] The power supply module 414 is used to supply power to the processor 401, memory 402, communication interface 403, display screen 409, camera 410, audio module 413 and other modules in the second electronic device 200.

[0186] Button 415 includes the power button, volume buttons, etc. Button 415 can be a mechanical button or a touch button.

[0187] Figure 4D is a schematic diagram of the hardware structure of another target object 300 provided in an embodiment of this application. It should be understood that the target object 300 shown in Figure 4D is an active device.

[0188] As shown in Figure 4D, the target object 300 includes the processor 401, memory 402, and communication interface 403 mentioned above, as well as a sensor module 416, camera 417, power supply module 418, and button 419.

[0189] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the target object 300. In other embodiments of this application, the target object 300 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0190] The functions of camera 417 and button 419 are described above in the section on camera 410 and button 415 in the second electronic device 200. Optionally, the target object 300 may not be equipped with camera 410 and / or button 419.

[0191] The power supply module 418 is used to supply power to modules such as the processor 401, memory 402, communication interface 403, and camera 417 in the target object 300. Optionally, the power supply module 418 can be used to connect to a battery or to receive charging input from a charger. The charger can be a wireless charger or a wired charger.

[0192] The target object 300 can establish a wireless communication connection with the second electronic device 200 through the communication interface 403.

[0193] Sensor module 416 may include, for example, pressure sensors and attitude sensors.

[0194] Pressure sensors are used to sense pressure signals and convert them into electrical signals. There are many types of pressure sensors, such as resistive pressure sensors and inductive pressure sensors. In some embodiments, the target object 300 detects pressure data through a pressure sensor and sends this pressure data to a second electronic device 200. Optionally, the second electronic device 200 can determine whether there is a touch operation on the display screen based on the pressure data, thereby determining whether it is necessary to obtain the position information of the target object 300's intended interaction location on the display screen.

[0195] An attitude sensor is used to detect the attitude data of the stylus. Optionally, the attitude sensor can be, for example, an inertial measurement unit (IMU). For example, the IMU can help the target object 300 determine its motion state and attitude by measuring dynamic information such as acceleration and angular velocity.

[0196] Optionally, the IMU includes an accelerometer and a gyroscope. The accelerometer in the IMU measures the acceleration of the target object 300 along three axes (typically the X, Y, and Z axes). Acceleration refers to the rate of change of the target object 300's velocity. The accelerometer primarily captures the linear acceleration of the target object 300 and is commonly used to detect the target object 300's movement, tilt, or vibration. The gyroscope can be used to determine the target object 300's attitude. For example, the gyroscope provides rotational information of the target object 300, helping to calculate its attitude changes. In some embodiments, the gyroscope can determine the angular velocity of the target object 300 around the three axes (typically the X, Y, and Z axes), i.e., the rate of rotation. Optionally, the IMU may also include a magnetometer, which measures the direction of the Earth's magnetic field, providing an azimuth angle. This is typically combined with the detection data from the accelerometer and gyroscope to calculate the target object 300's attitude. By measuring the direction of the Earth's magnetic field, the IMU can provide relatively accurate orientation data.

[0197] In some embodiments, the target object 300 detects attitude data via an attitude sensor and sends the attitude data to the second electronic device 200. Optionally, the second electronic device 200 can obtain correction information based on the attitude data. This correction information is used to obtain more accurate, corrected position information during the process of determining the intended interactive position of the target object 300 on the display screen, thereby eliminating the influence of the user's hand gesture on the position information. In other embodiments, the second electronic device 200 can predict the motion trajectory of the target object 300 based on the attitude data. For example, the second electronic device 200 can predict the handwriting using the attitude data when displaying handwriting corresponding to the target object 300.

[0198] The following section uses the first electronic device 100 as the positioning device, the second electronic device 200 as the smart screen, and the target object 300 as a finger or stylus as an example to describe in detail the interaction method provided in the embodiments of this application.

[0199] In some embodiments, users can control the smart screen by touching it with a stylus or finger, enhancing the user's interactive experience. Therefore, the smart screen also needs to detect the touch location of the stylus or finger to determine the corresponding touch event. For example, the touch event could be a control trigger event or a handwriting display event.

[0200] In some examples, the smart screen uses a positioning device to determine the touch location of a stylus or finger. Optionally, the positioning device is installed on the smart screen. Optionally, the positioning device can be pre-configured in the smart screen, or the user can choose to configure the positioning device separately. For example, the positioning device can be pre-embedded in the smart screen and integrated with the existing modules in the smart screen. For example, the positioning device can be integrated with the smart screen's camera or microphone modules into a single module. Alternatively, the positioning device can be a standalone accessory, which the user can choose to install or not. Optionally, the positioning device can be installed on the smart screen using various methods such as clip fixing, magnetic fixing, or lifting fixing.

[0201] For example, as shown in FIG5, the positioning device 51 is installed above the smart screen, for example, it can be located next to the camera 52 of the smart screen.

[0202] In some examples, the positioning device can be a positioning device based on technologies such as direct time-of-flight (DTOF) sensors, indirect time-of-flight (ITOF) sensors, binocular sensors, and structured light sensors. Optionally, the DTOF sensor can be, for example, a rotating single-wire sensor. The above types of sensors can be used to construct radar positioning devices.

[0203] The working principle of the DTOF sensor is based on time of flight (TOF) technology. It calculates the distance from the object to the sensor by measuring the time difference between the emission of a light pulse (such as laser or infrared light) from the sensor and its reflection back from the surface of the object.

[0204] The working principle of an ITOF sensor is similar to that of a DTOF sensor, both being based on TOF technology. However, an ITOF sensor calculates distance by measuring the phase difference of light pulses, rather than directly measuring the round-trip time.

[0205] Binocular sensors mimic the human visual system, using two cameras to capture images of the same scene from different perspectives, thereby calculating the depth information of objects in the scene.

[0206] Structured light sensors acquire depth information by projecting a known light pattern onto the surface of an object and analyzing the deformation of that pattern.

[0207] In some embodiments, the positioning device can determine the touch position of a stylus or finger on the smart screen based on light reflection information. Optionally, the reflectivity of the target object, such as a stylus or finger, is greater than or equal to a preset threshold, for example, 40%. Thus, when the positioning device emits a laser in a direction parallel or approximately parallel to the display screen of the smart screen, the laser can be reflected by the stylus or finger performing touch operations on the smart screen. Then, as shown in Figure 5, the positioning device 51 can determine data information about the position of the reflected laser on the stylus or finger relative to the positioning device, such as distance and angle information, based on the light reflection information.

[0208] Optionally, the positioning device 51 is a miniaturized accessory with low cost and easy installation. Compared with the sensing devices used to implement smart screen touch technology in the prior art, it can reduce hardware costs. Since it does not need to photograph the entire display screen, there is no need to configure long arm components, which improves the overall aesthetics of the product. Compared with other implementation methods, this solution has less stringent installation requirements and is easier to perform subsequent calibration, which also improves the convenience and operability of use.

[0209] It should be understood that this application uses DTOF radar as an example to describe the process of DTOF radar locating a target by emitting laser light. However, DTOF radar or other types of positioning devices can also locate a target in other ways, which will not be elaborated upon in this application. For example, DTOF radar can also locate a target by emitting infrared light or other light beams.

[0210] The positioning device emits a laser on a plane parallel (or nearly parallel) to and slightly higher than the touchscreen surface to detect potential user touch operations on the display. The location of this reflected laser is, for example, approximately the actual touch position of a stylus or finger on the touchscreen. For ease of description, the actual touch position that the smart screen needs to determine will be described as the target position. In some cases, the smart screen uses the location of this reflected laser as the target position.

[0211] For example, as shown in Figure 6, the positioning device includes a processing module, a ranging and angle measuring module, and a wired communication module. The processing unit is used to schedule the operation of the entire positioning device; for example, it can send data output from the ranging and angle measuring module to a smart screen via the wired communication module. The positioning device and the smart screen are connected via a wired communication module. Optionally, the positioning device can receive power from the smart screen through this wired communication connection. It should be understood that the positioning device and the smart screen can also establish a wireless communication connection via a wireless communication module. The processor is also used to schedule the positioning device to emit laser light. The ranging and angle measuring module can be used to obtain the distance and angle information of the target position relative to the positioning device based on laser reflection information.

[0212] Optionally, the distance information of the target location includes the distance between the target location and the positioning device. The angle information of the target location includes the offset angle of the target location relative to the positioning device.

[0213] Optionally, the laser reflection information may include, for example, reflection time information and reflection location information.

[0214] Optionally, the positioning device can also send the detected laser reflection information to the smart screen, which will then determine the distance and angle information of the target position relative to the positioning device.

[0215] For example, the positioning device emits a laser and can acquire the reflected light formed by the reflection of the laser. Then, the positioning device can determine time information based on the time of laser emission and the time of receiving the reflected light. The positioning device can also determine the angle information of the target object relative to the positioning device based on the emitted laser and the received reflected light. It should be understood that the reference direction used by the positioning device in determining the angle information can be any predefined direction. For example, this reference direction could be parallel to the top edge of the smart screen, perpendicular to the top edge of the smart screen, etc.

[0216] For example, the positioning device determines the distance information of the target object relative to the positioning device based on the time information. Then, the positioning device sends the distance and angle information to the smart screen to trigger the smart screen to determine the location information of the target based on the distance and angle information.

[0217] For example, a positioning device sends time and angle information to a smart screen, triggering the smart screen to determine the target's location based on the time and angle information. The following section describes the interaction process between the positioning device and the smart screen, using the example of the smart screen receiving distance and angle information from the positioning device.

[0218] Thus, by simply adding a small positioning accessory (such as a positioning device) to the smart screen, without any other additional hardware support, the location of the touch screen can be determined, thus meeting the user's interaction needs between the target object and the smart screen.

[0219] Furthermore, based on the positioning device, users can perform touch operations on the smart screen using various types of target objects, meeting users' flexible usage needs for the smart screen.

[0220] In some embodiments, the smart screen is configured with the location information of a positioning device, such as the position information of the positioning device relative to the smart screen. After obtaining the distance and angle information of the target position relative to the positioning device sent by the positioning device, the smart screen can determine the pixel position of the target position on the display screen, such as its x-axis and y-axis coordinates. This allows the smart screen to determine the touch event corresponding to the control that triggers the display at that pixel position, or to display handwriting at that pixel position. For example, during the display of a whiteboard application, the smart screen receives the distance and angle information sent by the positioning device. Based on this information, the smart screen can determine the distance and offset angle of the target position of the touch operation relative to the positioning device, thereby determining the position information of the positioning device. Then, the smart screen can perform coordinate system transformation on the target position based on the positioning device's position information to determine the pixel position where handwriting needs to be displayed on the whiteboard application interface, and then instruct the whiteboard application to display the corresponding handwriting.

[0221] In some examples, the smart screen is pre-configured with the location information of the positioning device, or the smart screen can receive the user's location calibration operation to obtain the location information of the positioning device.

[0222] For example, the positioning device is a pre-installed module in the smart screen, so users do not need to install a separate positioning device. Therefore, the location information of the positioning device can be pre-configured in the smart screen.

[0223] For example, the smart screen may have pre-defined installation locations for positioning devices. Users simply need to install the positioning device at those locations. The smart screen can then be pre-configured with the location information for these installation locations, which can then be used as the location information for the positioning device.

[0224] For example, as shown in Figure 7, after the positioning device is installed, the smart screen can trigger the positioning device's position calibration process based on user operations. The smart screen has pre-configured position information for the four corners of the screen. Therefore, after the positioning device is installed, the smart screen can instruct the user to perform touch operations at these four positions. Based on the distance and angle information output by the positioning device, the position information of the positioning device can be calculated in reverse, completing the position calibration process. It should be understood that the positioning positions are not limited to the four corners of the display screen.

[0225] In some examples, the positioning device can obtain its position information relative to the smart screen. After obtaining the distance and angle information of the target location, the positioning device can also directly obtain the target location's position information based on its own position information. Then, the positioning device can send this target location information to the smart screen. The smart screen can then trigger corresponding touch events based on the received position information. In other words, the target location's position information can be determined by either the smart screen or the positioning device. Alternatively, other devices (such as servers) can also obtain the positioning device's position information, as well as the distance and angle information sent by the positioning device, to determine the target location's position information and send it to the smart screen.

[0226] In this way, based on the location information of the positioning device, the distance and offset angle of the target position of the touch operation relative to the positioning device are determined through laser reflection information, thus simplifying the location information determination process. There is no need to add additional complex sensing devices to the smart screen, reducing the hardware cost of the smart screen.

[0227] In some embodiments, the positioning device emits a laser beam. Other objects in the environment (such as speakers, plants, or other interfering objects near the smart screen in a home environment that are close to the screen but not within the detection area) may also emit the laser beam, forming a reflected beam. Subsequently, after acquiring distance and angle information, the smart screen can filter the received distance and angle information according to the acquired detection range to retain the distance and angle information corresponding to the interaction between the target object and the smart screen. Then, based on the filtered distance and angle information, the smart screen can determine the target object's position on the display screen and execute the corresponding interactive event.

[0228] For example, a smart screen displays a full-screen interactive interface to receive interactive operations from target objects on the full-screen interface. The smart screen can then, based on the size range of the display, discard the distance and angle information generated by reflected light beams from objects outside that size range, retaining only the distance and angle information generated by reflected light beams from target objects within that size range, thus determining the target's location.

[0229] For example, a smart screen displays interactive applications (such as whiteboard applications), and receives interactive operations from target objects through the window of the application. The smart screen can then, based on the size range of the window, discard the distance and angle information generated by reflected light from other objects outside that size range, retaining only the distance and angle information generated by reflected light from target objects within that size range, thus determining the target's location.

[0230] For example, the smart screen can obtain the interactive area indicated by the user based on the user's operation. This interactive area could be, for instance, the area the user selects or circles on the screen before interacting. The screen then receives the interaction from the target object through this interactive area. Based on the size range of the interactive area, the smart screen can discard the distance and angle information generated by the reflected light beams of other objects outside that size range, retaining only the distance and angle information generated by the reflected light beams of the target object within that size range, thus determining the target's location.

[0231] In this way, the smart screen can eliminate environmental interference, determine the interaction position between the target object and the smart screen, and provide users with a more accurate interactive experience.

[0232] In some embodiments, the positioning device emits laser beams at a high frequency to detect the distance and angle information of the target position of the stylus or finger relative to the positioning device. Optionally, the positioning device can also rotate to emit the laser beam. In this way, as the laser beam rotates, single-line ranging can be changed to scanning of the laser surface.

[0233] Optionally, the positioning device includes a motor and a rotating mirror to rotate the laser beam. Optionally, the positioning device is pre-configured with a rotation angle to ensure that the resulting laser scanning plane covers the entire display screen. Optionally, the positioning device is typically installed at an edge or corner of the smart screen, such as the center of the top bezel; alternatively, it could be located on another bezel of the smart screen. This rotation angle can be 360 ​​degrees, and the positioning device or the smart screen can then detect the target object through data filtering. Alternatively, the rotation angle can be less than or equal to 180 degrees, which can reduce the amount of captured data, but this places certain requirements on the design of the rotation actuator of the positioning device.

[0234] The rotating mirror is used to scan the laser beam, enabling multi-angle depth measurement by the positioning device. For example, DTOF radar typically can only directly measure depth information in front of it. To expand the working range of DTOF radar and avoid being limited to a fixed angle, the rotating mirror is used to dynamically change the direction of the laser beam. By rotating the mirror, the laser beam can illuminate the object surface from different angles, thus enabling the scanning of a wider area.

[0235] For example, as shown in Figure 8, during the high-frequency laser beam emission process, the positioning device rotates the laser beam through a motor and a rotating mirror, thereby forming a fan-shaped laser scanning plane on the surface of the smart screen.

[0236] Alternatively, the positioning device can also rotate the laser beam by directly rotating the laser-emitting module.

[0237] Thus, based on laser scanning plane positioning, the stylus or finger can be moved to any position on the smart screen's display to pinpoint the target location, resulting in more precise touch response. For example, the positioning device can achieve a positioning accuracy within ±3 millimeters. Positioning accuracy measures the deviation between the target position and the stylus tip or finger's position on the display screen.

[0238] In some embodiments, if the positioning device sends a laser beam at a sufficiently high frequency and rotates the laser beam quickly enough, it can acquire dense, high-refresh-rate point cloud data reflected from the target location of the stylus or finger. Then, by using subsequent algorithmic point selection and clustering, the positioning device can obtain more accurate distance and angle information for the target location.

[0239] Point cloud data is a collection of numerous data points with spatial locations (usually three-dimensional coordinates). These data points are typically acquired using technologies such as laser scanning, LiDAR, structured light, depth cameras, or stereo vision. Each point represents a location in three-dimensional space and can be accompanied by additional attributes such as color, intensity, and reflectivity.

[0240] For example, after the laser beam emitted by the DTOF radar scans the surface of the stylus, the DTOF radar can receive a large amount of reflected light information, such as point cloud data.

[0241] In some examples, the stylus or finger has a certain width, causing the laser to only contact the surface of the stylus or finger, and not its center. However, the actual touch position of the stylus or finger on the smart screen's display is much closer to this center. For example, the touch position of the stylus on the smart screen's display is at the tip of the stylus, which is located on the central axis of the stylus's shaft. Therefore, after acquiring point cloud data, the positioning device can use algorithms to select points and cluster them to obtain the true target position on the central axis, thus obtaining more accurate distance and angle information.

[0242] For example, as shown in Figure 9(a), the positioning device emits a high-frequency, rotating laser beam to form a fan-shaped laser scanning plane. When a user uses a stylus to perform touch operations on the smart screen, the laser beam scans the outer surface of the stylus. The positioning device can then receive point cloud data reflected from the outer surface of the stylus. For example, the laser beam may consist of five laser beams, and the hollow dots in the figure represent the point cloud data acquired by the positioning device in one frame. Therefore, if the laser beam rotates at 60 Hz, the positioning device can acquire 60 sets of such point cloud data per second. It should be understood that the embodiments of this application do not limit the number of laser beams included in the laser beam.

[0243] Subsequently, as shown in Figure 9(b), since the laser beam illuminates the outer surface of the stylus at multiple points, it may be impossible to accurately determine the actual touch position of the stylus tip on the display screen. Furthermore, arbitrarily selecting one point cloud as the target position may result in a significant deviation between the target position and the actual touch position. Therefore, the positioning device can filter the acquired point cloud data based on the stylus's outline and point cloud data, and then use algorithms such as filtering and clustering to obtain the centroid of the scanned stylus shaft, thereby more accurately obtaining the distance and angle information corresponding to the true target position of the stylus tip. For example, as shown in Figure 9(b), hollow dots represent the original point cloud data, and solid dots represent the target position after clustering and selection.

[0244] In this way, by selecting points and clustering them using algorithms, the distance and angle information of the actual target location can be estimated, which makes it easier to obtain more accurate target location information and achieve more accurate touch response.

[0245] In some embodiments, the laser scanning plane should be as close as possible to the surface of the smart screen's display, so that the calculated target position is as close as possible to the actual touch position of the stylus or finger on the display. In some cases, the surface of the display is not perfectly flat and may have unevenness; the laser scanning plane should be slightly higher than the highest point on the display to avoid the laser scanning plane scanning the display itself. Therefore, there is a possibility that the stylus or finger has touched the laser scanning plane before touching the display, causing the positioning device to output distance and angle information of the target position, providing the user with a hovering touch experience. Alternatively, there is another possibility that the stylus or finger has left the display (e.g., not touching the display) but has not left the laser scanning plane (e.g., still touching the laser scanning plane), leading to inconsistent perception of the lift height by the user. These situations have a relatively small impact on click and swipe-type touch operations. For example, the user may only feel the need to lift the stylus or finger to a higher position to avoid triggering a touch event on the smart screen. However, for scenarios such as writing or drawing, issues such as ink leakage and scribbling may occur, affecting the user experience.

[0246] For example, as shown in Figure 10(a), a top view of the smart screen reveals that its surface is not perfectly flat; some areas are higher than others, and the laser scanning plane is slightly higher than the highest point of the display. Therefore, when a stylus needs to touch a lower area of ​​the display, it may actually touch the laser scanning plane but not the display itself. In this situation, if the positioning device outputs distance and angle information of the target location to the smart screen, causing it to display the corresponding handwriting, problems such as ink leakage and smearing may occur.

[0247] Optionally, the target object can be equipped with a pressure sensor. The pressure sensor's detection data determines whether the object has actually touched the display screen. Then, the smart screen is triggered to respond to the corresponding touch event, thus improving the user experience.

[0248] For example, as shown in Figure 11, the stylus is equipped with a pressure sensor that can detect pressure data from the pen tip. Optionally, after detecting pressure data through the pressure sensor, the stylus can send the pressure data to the smart screen via a wireless or wired communication module. Optionally, the stylus can also send the pressure data to a positioning device, which will then calculate or forward it to the smart screen.

[0249] In this way, after receiving distance and angle information from the target location, the smart screen can combine this information with pressure data to determine whether to enable touch events. For example, if the pressure data is greater than or equal to a preset pressure threshold, the smart screen can determine that the stylus is touching the display screen and enable handwriting. Alternatively, if the pressure data is less than the preset pressure threshold, the smart screen can determine that the stylus is not touching the display screen and disable handwriting.

[0250] Optionally, after acquiring pressure data, the positioning device can also determine whether it needs to acquire distance and angle information of the target location based on whether the pressure data is greater than or equal to a preset pressure threshold. In this way, the smart screen can acquire the location information of the target location and enable the corresponding touch events upon receiving the distance and angle information of the target location, without needing to determine again whether to enable the touch events.

[0251] Optionally, the smart screen can also adjust the thickness of the handwriting based on changes in the received pressure data, thereby providing users with a better interactive experience.

[0252] Optionally, the stylus can also be equipped with pressure controls, physical buttons, proximity sensors, etc., to trigger the smart screen to respond to corresponding touch events when the stylus is used for touch operations. That is, the functions of the pressure sensor mentioned above can also be achieved through these implementation methods, without being limited to the implementation method of the pressure sensor.

[0253] For example, as shown in Figure 10(b), when the stylus tip touches the display screen at position A, the stylus sends the pressure data detected by the pressure sensor to the smart screen via a positioning device. The stylus also touches the laser scanning plane. The positioning device can determine the time (or distance) and angle information for calculating the target position based on the detected point cloud data and send this information to the smart screen. The smart screen, combining the pressure data obtained (greater than or equal to a preset pressure threshold), can determine the target position's location information based on the received time (or distance) and angle information, and display the handwriting at position A corresponding to that location, thus achieving effective writing. Afterwards, the user moves the stylus from position A to position B. At position B, the stylus tip does not touch the display screen, and the pressure sensor does not detect any pressure data. However, at position B, the stylus has touched the laser scanning plane, and the positioning device can determine the time and angle information of the target position based on the detected point cloud data and send it to the smart screen. Therefore, if the smart screen combines the current situation of not receiving pressure data, it does not need to calculate the location information of the target position, nor does it need to display the handwriting, thus avoiding invalid writing.

[0254] In this way, by combining pressure data, the system can provide users with a realistic writing experience on the display screen using a stylus, avoiding issues such as ink leakage and scribbling.

[0255] In some embodiments, when a user holds a target object and touches it on the smart screen, the actual touch position on the screen may deviate from the target position determined on the laser scanning plane due to the posture of the held object. Therefore, the impact of this deviation on the user experience can be reduced by incorporating the object's posture data.

[0256] For example, as shown in Figure 12, when a user holds the stylus, the stylus may tilt slightly. For instance, the actual touch position of the stylus tip on the display screen might be slightly to the right, while the touch position (i.e., the target position) between the laser scanning plane and the stylus body might be slightly to the left, resulting in a discrepancy between the two positions. Therefore, directly using the touch position of the stylus body as the actual touch position could lead to problems in the final touch event response, such as the handwriting display position not matching the user's actual touch position, thus affecting the user experience.

[0257] Optionally, as shown in Figure 13, the target object (such as a stylus) is equipped with an attitude sensor, which is used to detect the attitude data of the stylus. Optionally, the attitude sensor can be, for example, an inertial measurement unit (IMU). For example, the IMU can help determine the motion state and attitude of the target object by measuring dynamic information such as acceleration and angular velocity.

[0258] Optionally, after the target object detects attitude data via the attitude sensor, it can send the attitude data to the smart screen. Alternatively, the target object can send the attitude data to the smart screen via a positioning device, or it can send the attitude data directly to the smart screen. Correspondingly, in determining the target's location, the smart screen can combine the attitude data with the acquired distance and angle information to obtain a more accurate location. It should be understood that this target location can be interpreted as the touch position of the target object on the display screen, as determined by the smart screen.

[0259] Optionally, the posture data includes acceleration data, and the smart screen can also combine this acceleration data to achieve handwriting prediction.

[0260] In this way, by combining posture data, the smart screen can provide users with a more realistic touch experience and avoid positional deviations that could affect user experience.

[0261] In some embodiments, the positioning device can detect multi-touch operations by a user on a display screen by emitting a rotating laser beam. That is, the positioning device can send information about the multiple target locations corresponding to the multiple touch points of the user's multi-touch operation to the smart screen. However, since the positioning device obtains the target location of the object by emitting a laser beam, if there are multiple touch points located radially to the laser beam in a multi-touch operation, some touch points may be obstructed and cannot be accurately identified. Therefore, the target object recognition problem in multi-touch scenarios can also be solved by setting up multiple positioning devices or combining them with other positioning methods.

[0262] In some embodiments, the scanning surface for locating the target position can be formed by mechanical scanning as described above, or by electrical scanning. For example, as mentioned above, the positioning device emits a light beam, and the emission direction of the beam is changed by a rotating mirror to form a scanning surface. Alternatively, a motor drives the emission module to rotate to form the scanning surface of the beam. As another example, the positioning device can also be implemented using electrically scanned solid-state lidar technology. Electrically scanned solid-state lidar technology is a lidar technology that uses electronic control to scan a laser beam to achieve environmental perception. Unlike traditional mechanically scanned lidar, electrically scanned solid-state lidar has no moving parts; instead, it relies on electronic methods (such as electro-optic modulation or beam deflection) to rapidly scan the laser beam, thereby achieving high-precision ranging and imaging in space.

[0263] For example, as shown in Figure 14(a), a multiple-input multiple-output (MIMO) positioning device is used to form a linear array or area array on the surface of the smart screen display. In this way, during touch operation, the positioning device can obtain the touch position of the target object on the linear array or area array, thereby determining the distance information of the target position. It should be understood that, optionally, the shadow corresponding to the linear array or area array shown in Figure 14(a) can cover the entire display screen; the shadow in the figure is only for illustrative purposes, and the linear array or area array can completely scan the bottom of the entire display screen.

[0264] As shown in Figure 14(a), the positioning device and smart screen are viewed from above, following the direction of the arrows.

[0265] If the positioning device uses an electrically scanned solid-state lidar technology to emit lasers in a linear array, as shown in the top view of Figure 14(b), this linear array can completely cover the entire display screen, thereby enabling the positioning of a target object at any location on the display screen for interactive operation. The target object's location information acquired by the smart screen includes its x-axis coordinate and y-axis coordinate.

[0266] If the positioning device uses an electronically scanned solid-state LiDAR to emit lasers in the form of an array, as shown in the top view of Figure 14(c), this array can completely cover the entire display screen, thus enabling the positioning of a target object at any location on the screen for interactive operation. Furthermore, the positioning device can also acquire the external contour information of the target object through this array, including height information. Therefore, the location information of the target obtained by the smart screen can subsequently include not only the x-axis and y-axis coordinates but also the z-axis height information, enabling richer spatial interaction scenarios.

[0267] In this way, the location of a target in space can be perceived through electronically scanned solid-state lidar technology.

[0268] Furthermore, electronically scanned solid-state lidar overcomes the drawback of single-beam lasers requiring mechanical scanning through MIMO technology. It is not limited by mechanical rotation speed, has a higher frame rate, and can achieve more accurate target position determination, but the implementation cost is relatively high.

[0269] Figure 15 is a flowchart illustrating an interaction method provided in an embodiment of this application. It should be noted that this method is not limited to the specific order described in Figure 15 and below. It should be understood that in other embodiments, the order of some steps in this method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0270] S1501, The first electronic device emits a light beam to form a scanning plane at a position close to the display screen of the second electronic device, the scanning plane being parallel or approximately parallel to the plane of the display screen.

[0271] The first electronic device is installed independently on the second electronic device, or the first electronic device is integrated into the second electronic device.

[0272] For example, the first electronic device can be integrated with the camera or microphone modules of the second electronic device into a single module.

[0273] Optionally, the first electronic device is located at the center of the upper frame of the second electronic device. The center position includes either the exact center or a position near the exact center.

[0274] For example, as shown in FIG5, the positioning device 51 (such as a first electronic device) is mounted on the upper frame of the smart screen (such as a second electronic device), for example, it may be located next to the camera 52 of the smart screen.

[0275] The beam is either a laser beam or an infrared beam.

[0276] The goal of beam scanning is to create a scanning plane that is parallel to the plane of the display screen. However, due to limitations such as the installation errors of the first electronic device, the scanning plane may only be approximately parallel to the plane of the display screen.

[0277] S1502, The first electronic device acquires the reflected beam formed by the reflection of the beam by the target object.

[0278] The reflected light beam is used to determine the positional information of the target object on the display screen at the intended interaction location. This intended interaction location includes, for example, the touch position of the target object on the display screen, or the position corresponding to a hover touch on the display screen—the on-screen position where the user intends to interact with the second electronic device through the target object. In some examples, the position of the reflected light beam on the target object approximates this intended interaction location.

[0279] In some embodiments, the first electronic device is configured with a rotation module. The rotation module is used to form a scanning plane.

[0280] Optionally, the rotation module may include, for example, a rotating mirror. Exemplarily, as shown in FIG8, the positioning device (such as a first electronic device) rotates the beam via a rotating mirror during the high-frequency emission of the beam, thereby forming a fan-shaped scanning plane on the display surface of the smart screen (such as a second electronic device).

[0281] Optionally, the rotation module may include, for example, a motor for rotating the transmitting module, such that as the transmitting module rotates, the beam emitted by the transmitting module forms a scanning plane.

[0282] In this way, a scanning plane is formed through mechanical scanning, thereby enabling the positioning of the target object.

[0283] In other embodiments, the first electronic device is configured with a transmitting module and a receiving module, the transmitting module including multiple transmitting units and / or the receiving module including multiple receiving units. The multiple transmitting units are used to emit a light beam forming a scanning plane. The multiple receiving units are used to receive reflected light beams.

[0284] For example, as shown in FIG14(b), the first electronic device emits lasers through multiple emitting units to form a laser linear array. Also for example, as shown in FIG14(c), the first electronic device emits lasers through multiple emitting units to form a laser area array.

[0285] Thus, through MIMO electrical scanning, a scanning plane is formed, enabling the positioning of the target object.

[0286] In some embodiments, the target object is an active device or a passive object. Active devices include styluses or touch gloves, while passive objects include fingers. And / or, the first electronic device is any one of DTOF radar, ITOF radar, binocular radar, structured light radar, and electrically scanned solid-state lidar. And / or, the second electronic device is a smart screen. The smart screen can be a screen for home or office use.

[0287] S1503, the second electronic device executes the interactive operation corresponding to the position information based on the position information of the target object indicated by the reflected beam on the display screen at the intended interactive position.

[0288] In some embodiments, the location information includes first location information. The second electronic device performs user touch operations and / or presents corresponding display elements on the second electronic device based on the first location information.

[0289] Display elements include, for example, notes or animations.

[0290] For example, in response to a touch of a target object at a certain location on the display screen, the second electronic device can obtain the first location information corresponding to the touch, thereby determining the control corresponding to the first location information, triggering the control, and executing the user's touch operation. Alternatively, the second electronic device can display a corresponding display element at the location corresponding to the first location information. This display element may be, for example, an animation or display component that appears along with the user's interaction intention, or handwriting, etc.

[0291] In other embodiments, the location information includes multiple second location information. Based on the multiple second location information, the second electronic device displays a trajectory corresponding to the user's interactive operation.

[0292] For example, in response to a touch operation that moves a target object on a display screen, the second electronic device can acquire multiple associated second position information, thereby displaying the corresponding trajectory. For instance, the second electronic device can display corresponding drawing or writing handwriting.

[0293] In this way, the second electronic device can achieve flexible display in different usage scenarios.

[0294] In some embodiments, before executing the interactive operation corresponding to the position information of the target object indicated by the reflected beam on the display screen, the second electronic device can also obtain relevant information corresponding to the reflected beam sent by the first electronic device. This relevant information can be distance and angle information corresponding to the target object's intended interaction position. Alternatively, it can be time and angle information corresponding to the target object's intended interaction position. Or, it can be the position information of the target object's intended interaction position. In other words, the interaction between the first and second electronic devices based on different information allows them to determine the steps to be executed. This application embodiment does not limit the entity executing the relevant information acquisition process. The execution process of the first and second electronic devices is described in detail below.

[0295] The intended interaction location can be understood, for example, as the target location described above. The reflection position of the light beam on the target object approximates the intended interaction location of the target object on the display screen.

[0296] In some embodiments, the first electronic device includes a transmitting module, a receiving module, a processing module, and an angle measurement module. The transmitting module is used to transmit a light beam. The receiving module is used to receive a reflected light beam. The processing module is used to acquire first-dimensional information based on the time of transmitting the light beam and the time of receiving the reflected light beam. The angle measurement module is used to measure second-dimensional information about the intended interaction position relative to the first electronic device. The first-dimensional information and the second-dimensional information are used to determine position information.

[0297] The angle measurement module is capable of measuring the dimensional information of the intended interaction position relative to the angle measurement module. The angle measurement module can treat this dimensional information as a second dimension of the intended interaction position relative to the first electronic device.

[0298] Optionally, the first dimension information may be, for example, time information, and the second dimension information may be, for example, angle information. The time information is used to determine the distance information of the intended interaction location relative to the first electronic device. Alternatively, the first dimension information may be, for example, distance information, and the second dimension information may be, for example, angle information.

[0299] Optionally, the first electronic device is configured to: send first dimension information and second dimension information to the second electronic device. The second electronic device is configured to: determine location information based on the first dimension information and the second dimension information.

[0300] For example, after obtaining the distance and angle information of the intended interaction location relative to itself, the first electronic device can send the distance and angle information to the second electronic device. Accordingly, the second electronic device can determine the position information of the intended interaction location on the display screen based on the received distance and angle information.

[0301] For example, after acquiring time information and angle information of the intended interaction location relative to the first electronic device, the first electronic device can send the time information and angle information to the second electronic device. Accordingly, the second electronic device can determine the distance information of the intended interaction location relative to the first electronic device based on the received time information related to the transmission and reception times. Then, the second electronic device can determine the position information of the intended interaction location on the display screen based on the distance information and angle information.

[0302] Thus, based on the position information of the first electronic device, the distance and offset angle of the intended interaction position relative to the first electronic device are determined by the reflection of the light beam, thereby determining the position information of the intended interaction position. This simplifies the process of determining position information and eliminates the need to add complex sensing devices to the second electronic device, thereby reducing the hardware cost of the second electronic device.

[0303] In some embodiments, the second electronic device is configured to: filter information based on first-dimensional information and / or second-dimensional information according to information associated with an interactive area on the display screen to obtain location information.

[0304] Optionally, the interactive area can be any of the following: full screen, application window, or area selected by the user. Optionally, the number of interactive areas can be one or more. For example, if multiple users select their own interactive areas, then there can be multiple interactive areas, and correspondingly, there can also be multiple targets for implementing the interaction.

[0305] Optionally, the associated information may include, for example, the size and location information of the interactive area.

[0306] For example, the second electronic device displays a full-screen interactive interface to receive interactive operations from the target object on the full-screen interface. That is, the interactive area is the full-screen interactive interface. Then, the second electronic device can, based on the size range of the display screen, discard the distance and angle information generated by the reflected light beams of other objects that are outside that size range, and only retain the distance and angle information generated by the reflected light beams of the target object within that size range, thus determining the target's position information.

[0307] For example, a second electronic device displays an interactive application (such as a whiteboard application), through which it receives interactive operations from the target object. The second electronic device can then, based on the window's size range, discard the distance and angle information generated by reflected light beams from other objects outside that range, retaining only the distance and angle information generated by reflected light beams from the target object within that range, thus determining the target's location.

[0308] For example, the second electronic device acquires the interactive area indicated by the user based on the user's operation, and receives the interactive operation of the target object through this interactive area. Then, the second electronic device can, based on the size range of the interactive area, discard the distance and angle information generated by the reflected light beams of other objects that are outside the size range, and only retain the distance and angle information generated by the reflected light beams of the target object within the size range, thereby determining the position information of the target.

[0309] In this way, the second electronic device can eliminate environmental interference, determine the interaction position between the target object and the second electronic device, and provide users with a more accurate interactive experience.

[0310] In some embodiments, the target object is an active device, which includes a pressure sensor. The target object is configured to: transmit pressure data detected by the pressure sensor to a second electronic device. The second electronic device is further configured to: perform an interactive operation corresponding to the location information based on the received pressure data.

[0311] Specifically, based on the received pressure data, the interactive operation corresponding to the location information is executed, including: after the second electronic device determines that the received pressure data is greater than or equal to a preset pressure threshold, the interactive operation corresponding to the location information is executed.

[0312] For example, as shown in Figure 11, a pressure sensor is configured in the stylus (such as the target object) to detect pressure data from the pen tip. Optionally, after detecting pressure data through the pressure sensor, the stylus can transmit the pressure data to a positioning device (such as a first electronic device) via a wireless communication module. Correspondingly, the positioning device receives the pressure data via the wireless communication module. Optionally, the positioning device forwards the pressure data to a smart screen (such as a second electronic device). For example, the positioning device sends the pressure data to the smart screen via a wired communication module. Alternatively, the stylus can also directly send the pressure data to the smart screen.

[0313] In this way, after receiving the first and second dimension information of the target location, the smart screen can combine the pressure data to determine whether interactive operations need to be enabled. For example, if the pressure data is greater than or equal to a preset pressure threshold, the smart screen can determine that the stylus is touching the display screen and enable handwriting. Alternatively, if the pressure data is less than the preset pressure threshold, the smart screen can determine that the stylus is not touching the display screen and disable handwriting.

[0314] In this way, by combining pressure data, the system can provide users with a realistic writing experience on the display screen using the target object, thus avoiding problems such as ink leakage and scribbling.

[0315] In some embodiments, the second electronic device is further configured to: adjust the presentation effect of the display element located at the first position information based on pressure data.

[0316] Optionally, rendering effects may include the thickness of the rendering path or the animation of the displayed elements.

[0317] In this way, the second electronic device can also adjust the thickness of the handwriting according to the changes in the received pressure data, thereby providing users with a better interactive experience.

[0318] In some embodiments, the target object is an active device, including an attitude sensor for acquiring attitude data of the target object. The attitude data is used to predict the handwriting position and / or to acquire correction information to determine position information.

[0319] In some examples, the second electronic device is also used to: receive attitude data; and, based on the attitude data, obtain correction information to determine position information.

[0320] Optionally, the attitude sensor can be, for example, an IMU. For instance, an IMU can help a target determine its motion state and attitude by measuring dynamic information such as acceleration and angular velocity.

[0321] For example, as shown in Figure 13, a stylus (such as a target object) is equipped with a posture sensor to detect the posture data of the target object. After the target object detects the posture data through the posture sensor, it can send the posture data to a smart screen (such as a second electronic device). Optionally, the target object can send the posture data to the smart screen through a positioning device, or it can send the posture data directly to the smart screen. Accordingly, in the process of determining the location information of the intended interaction position, the second electronic device can combine the posture data with the acquired first-dimensional information and second-dimensional information to obtain more accurate location information of the intended interaction position. For example, the second electronic device obtains location information A based on the first-dimensional information and second-dimensional information. Due to the tilt of the target object, the location indicated by location information A deviates from the user's intended interaction position. Then, the second electronic device can obtain correction information through the posture data, and then use the correction information to correct location information A to obtain location information B that indicates a position closer to the user's intended interaction position.

[0322] Optionally, the posture data includes acceleration data, and the smart screen can also combine this acceleration data to achieve handwriting prediction.

[0323] In this way, by combining posture data, the second electronic device can provide users with a more realistic touch experience and avoid positional deviations that could affect user experience.

[0324] In some embodiments, the target object includes a plurality of target objects.

[0325] In some examples, the location information includes the location information of multiple target objects at multiple intentional interaction locations on the display screen. The second electronic device is configured to: execute multiple interactive operations corresponding to the location information of the multiple intentional interaction locations.

[0326] For example, multiple users each hold a stylus and perform touch operations on the display screen. A first electronic device can locate multiple styluses, and a second electronic device needs to respond to multiple interactive operations from these styluses. Each stylus is a target object. Optionally, the detection and tracking of these multiple target objects can be implemented by multiple first electronic devices, which can use the same positioning technology or different positioning technologies. Optionally, one or more first electronic devices can capture target information only within a specified range. Optionally, the second electronic device can also filter the interactive position information of corresponding target objects by using preset interactive ranges for different target objects.

[0327] For example, a user might use multiple fingers to perform multi-finger operations on a display screen. The first electronic device can locate these multiple fingers, and the second electronic device needs to respond to the various interactive behaviors of these fingers. Furthermore, the second electronic device can determine a combined interactive intent based on these multiple interactive behaviors, such as a specific multi-finger operation intent of the user. In this context, each finger is a target object.

[0328] The process for determining the location information of each target object can refer to the process for determining the location information of the target objects described above.

[0329] In this way, multi-touch functionality of the second electronic device is achieved through the detection of multiple target objects.

[0330] Optionally, the first electronic device is further configured to perform the steps performed by the positioning device in the above embodiments, the second electronic device is further configured to perform the steps performed by the smart screen in the above embodiments, and the target object is further configured to perform the steps performed by the stylus or finger in the above embodiments, so as to realize the interaction method provided in the embodiments of this application.

[0331] In some solutions, multiple embodiments of this application can be combined, and the combined solution can be implemented. Optionally, some operations in the processes of each method embodiment may be combined, and / or the order of some operations may be changed. Furthermore, the execution order between the steps of each process is merely exemplary and does not constitute a limitation on the execution order between steps; other execution orders are also possible. It is not intended to indicate that the execution order is the only possible order in which these operations can be performed. Those skilled in the art will conceive of various ways to reorder the operations described herein. In addition, it should be noted that the process details involved in one embodiment of this document are similarly applicable to other embodiments, or different embodiments may be combined.

[0332] Furthermore, some steps in the method embodiments can be equivalently replaced with other possible steps. Alternatively, some steps in the method embodiments may be optional and can be deleted in certain use cases. Or, other possible steps may be added to the method embodiments.

[0333] Furthermore, the various method embodiments can be implemented individually or in combination.

[0334] The interaction method provided by the embodiments of this application has been described in detail above with reference to Figures 5-15. The first electronic device and the second electronic device provided by the embodiments of this application are described in detail below with reference to Figures 16 and 17.

[0335] In one possible design, Figure 16 is a schematic diagram of the structure of a first electronic device provided in an embodiment of this application. As shown in Figure 16, the first electronic device 1600 may include a transmitting module 1601 and a receiving module 1602. The first electronic device 1600 can be used to implement the functions of the first electronic device 100 (such as a positioning device) involved in the above method embodiments.

[0336] Optionally, the transmitting module 1601 is used to support the first electronic device 1600 in performing S1501 in FIG15.

[0337] Optionally, the receiving module 1602 is used to support the first electronic device 1600 in performing S1502 in FIG15.

[0338] The transmitting module 1601 and the receiving module 1602 can be integrated into a single module, such as a transceiver module; or they can be two separate modules. Optionally, the transceiver module may include multiple receiving units and / or multiple transmitting units, which can be implemented by a transceiver or transceiver-related circuit components.

[0339] Optionally, the first electronic device 1600 may further include a processing module for obtaining first-dimensional information based on the time of emitting the beam and the time of receiving the reflected beam.

[0340] Optionally, the first electronic device 1600 may further include an angle measurement module for measuring second-dimensional information of the intended interaction position relative to the first electronic device. The first-dimensional information and the second-dimensional information are used to determine the position information.

[0341] Optionally, the operation and / or function of each unit in the first electronic device 1600 are respectively to implement the corresponding process of the interaction method described in the above method embodiments. All relevant content of each step involved in the above method embodiments can be referred to the functional description of the corresponding functional unit. For the sake of brevity, it will not be repeated here.

[0342] Optionally, the first electronic device 1600 shown in FIG16 may further include a storage unit (not shown in FIG16) storing a program or instructions. When the transmitting module 1601 and the receiving module 1602 execute the program or instructions, the first electronic device 1600 shown in FIG16 can execute the interaction method described in the above method embodiments.

[0343] The technical effects of the first electronic device 1600 shown in Figure 16 can be referred to the technical effects of the interaction method described in the above method embodiments, and will not be repeated here.

[0344] In addition to being in the form of the first electronic device 1600, the technical solutions provided in this application may also be functional units or chips in the first electronic device, or devices used in conjunction with the first electronic device.

[0345] In one possible design, Figure 17 is a schematic diagram of the structure of the second electronic device provided in an embodiment of this application. As shown in Figure 17, the second electronic device 1700 may include: a transceiver module 1701, a processing module 1702, and a display module 1703. The second electronic device 1700 can be used to implement the functions of the second electronic device 200 (such as a smart screen) involved in the above method embodiments.

[0346] Optionally, the transceiver module 1701 is used to support the second electronic device 1700 to interact with the first electronic device 1600.

[0347] Optionally, the processing module 1702 is used to support the second electronic device 1700 in executing S1503 in FIG15.

[0348] Optionally, the display module 1703 is used to support the second electronic device 1700 in realizing the display function.

[0349] Optionally, the transceiver module may include a receiving unit and a transmitting unit, and may be implemented by a transceiver or transceiver-related circuit components, and may be a transceiver or a transceiver module. The operation and / or function of each unit in the second electronic device 1700 are respectively to implement the corresponding flow of the interaction method described in the above method embodiments. All relevant content of each step involved in the above method embodiments can be referred to the functional description of the corresponding functional unit, and for the sake of brevity, it will not be repeated here.

[0350] Optionally, the second electronic device 1700 shown in FIG17 may further include a storage unit (not shown in FIG17) storing a program or instructions. When the transceiver module 1701, the processing module 1702, and the display module 1703 execute the program or instructions, the second electronic device 1700 shown in FIG17 can execute the interaction method described in the above method embodiments.

[0351] The technical effects of the second electronic device 1700 shown in Figure 17 can be referred to the technical effects of the interaction method described in the above method embodiments, and will not be repeated here.

[0352] In addition to being in the form of a second electronic device 1700, the technical solution provided in this application may also be a functional unit or chip in a second electronic device, or a device used in conjunction with a second electronic device.

[0353] This application also provides a chip system, including: a processor coupled to a memory, the memory being used to store programs or instructions, wherein when the program or instructions are executed by the processor, the chip system implements the methods in any of the above method embodiments.

[0354] Optionally, the chip system may contain one or more processors. These processors can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, an integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0355] Optionally, the chip system may contain one or more memories. The memory may be integrated with the processor or disposed separately from it; this application embodiment does not limit this. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or disposed separately on different chips. This application embodiment does not specifically limit the type of memory or the arrangement of the memory and processor.

[0356] For example, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0357] It should be understood that each step in the above method embodiments can be completed by integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0358] This application also provides a computer-readable storage medium storing a computer program. When the computer program is run on a computer, it causes the computer to perform the aforementioned steps to implement the interactive method described in the above embodiments.

[0359] This application also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the interactive method described in the above embodiments.

[0360] Additionally, this application also provides an apparatus. Specifically, the apparatus may be a component or module, and may include one or more processors and a memory connected together. The memory stores a computer program. When the computer program is executed by one or more processors, the apparatus performs the interactive methods described in the above-described method embodiments.

[0361] The apparatus, computer-readable storage medium, computer program product, or chip provided in the embodiments of this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0362] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0363] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, the division of the above functional modules is only used as an example. In practical applications, the above functions can be assigned to different functional modules as needed; that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0364] In the several embodiments provided in this application, it should be understood that the disclosed methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division, and there may be other division methods in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of modules or units may be electrical, mechanical or other forms.

[0365] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0366] Computer-readable storage media include, but are not limited to, any of the following: USB flash drive, external hard drive, ROM, RAM, magnetic disk or optical disk, and other media capable of storing program code.

[0367] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An interactive system, characterized by The system includes: a first electronic device and a second electronic device; The first electronic device is used for: A light beam is emitted to form a scanning plane at a position close to the display screen of the second electronic device, the scanning plane being parallel or approximately parallel to the plane on which the display screen is located; The reflected beam formed by the reflection of the light beam by the target object is acquired, and the reflected beam is used to determine the position information of the intended interactive position of the target object on the display screen; The second electronic device is used for: Based on the location information, execute the interactive operation corresponding to the location information.

2. The system of claim 1, wherein, The location information includes first location information; The second electronic device is used for: Based on the first location information, the user's touch operation is performed on the second electronic device and / or corresponding display elements are presented on the second electronic device.

3. The system of claim 1 or 2, wherein, The location information includes multiple second location information; The second electronic device is used for: Based on the plurality of second location information, a trajectory corresponding to the user's interactive operation is presented on the second electronic device.

4. The system of any of claims 1-3, wherein, The first electronic device includes a transmitting module, a receiving module, a processing module, and an angle measurement module; The transmitting module is used to emit the light beam; The receiving module is used to receive the reflected light beam; The processing module is used to obtain first-dimensional information based on the time of emitting the light beam and the time of receiving the reflected light beam; The angle measurement module is used to measure the second dimension information of the intentional interaction position relative to the first electronic device; wherein the first dimension information and the second dimension information are used to determine the position information.

5. The system according to claim 4, characterized in that, The first electronic device is used for: Send the first dimension information and the second dimension information to the second electronic device; The second electronic device is used for: The location information is determined based on the first dimension information and the second dimension information.

6. The system according to claim 4 or 5, characterized in that, The second electronic device is used for: Based on information associated with the interactive area on the display screen, information is filtered according to the first dimension information and / or the second dimension information to obtain the location information.

7. The system of any of claims 1-6, wherein, The first electronic device is equipped with a rotation module; The rotation module is used to form the scanning plane.

8. The system of any of claims 1-6, wherein, The first electronic device is configured with a transmitting module and a receiving module, wherein the transmitting module includes multiple transmitting units and / or the receiving module includes multiple receiving units; The plurality of emitting units are used to emit light beams that form the scanning plane; The plurality of receiving units are used to receive the reflected light beam.

9. The system of any of claims 1-8, wherein, The target object is an active device, including a pressure sensor; The target object is used for: Send the pressure data detected by the pressure sensor to the second electronic device; The second electronic device is also used for: Based on the received pressure data, perform the interactive operation corresponding to the location information.

10. The system according to claim 9, characterized in that, The second electronic device is also used for: Based on the pressure data, adjust the presentation effect of the display element of the second electronic device located at the first position information.

11. The system of claim 10, wherein, The presentation effects include the thickness of the presentation trajectory or the animation effects of the display elements.

12. The system of any of claims 1-11, wherein, The target object is an active device, and the target object includes an attitude sensor. The attitude sensor is used to acquire the attitude data of the target object, and the attitude data is used to predict the handwriting position and / or acquire correction information to determine the position information.

13. The system according to claim 12, characterized in that, The second electronic device is also used for: Receive the attitude data; Based on the attitude data, the correction information is obtained to determine the position information.

14. The system of any of claims 1-13, wherein, The target object includes multiple target objects.

15. The system of claim 14, wherein, The location information includes location information of the multiple target objects at multiple intentional interaction locations on the display screen; The second electronic device is used for: Based on the location information of the multiple intentional interaction locations, execute the multiple interactive operations corresponding to the location information of the multiple intentional interaction locations.

16. The system according to any one of claims 1-15, characterized in that, The first electronic device is installed independently on the second electronic device, or the first electronic device is integrated into the second electronic device.

17. The system according to claim 16, characterized in that, The first electronic device is located in the middle of the upper frame of the second electronic device.

18. The system according to any one of claims 1-17, characterized in that, The target object can be an active device or a passive object. The active device includes a stylus or a touch glove, and the passive object includes a finger. And / or, The first electronic device is any one of the following: direct time-of-flight (DTOF) radar, indirect time-of-flight (ITOF) radar, binocular radar, structured light radar, and electronically scanned solid-state lidar. And / or, The second electronic device is a smart screen.

19. The system of any one of claims 1-18, wherein, The beam is a laser beam or an infrared beam.

20. An interaction method, characterized in that, The method is applied to a first electronic device, and the method includes: A light beam is emitted to form a scanning plane near the display screen of the second electronic device, the scanning plane being parallel or approximately parallel to the plane on which the display screen is located; The reflected beam formed by the reflection of the light beam by the target object is obtained. The reflected beam is used to determine the position information of the intended interactive position of the target object on the display screen. The position information is used by the second electronic device to perform the interactive operation corresponding to the position information.

21. The method of claim 20, wherein, The first electronic device includes a transmitting module, a receiving module, a processing module, and an angle measurement module; The emitted beam includes: emitting the beam through the emitted module; The step of acquiring the reflected beam formed by the reflection of the light beam by the target object includes: receiving the reflected beam through the receiving module; The method further includes: The processing module obtains first-dimensional information based on the time of emitting the light beam and the time of receiving the reflected light beam. The angle measurement module measures the second dimension information of the intentional interaction position relative to the first electronic device; wherein the first dimension information and the second dimension information are used to determine the position information.

22. The method of claim 21, wherein, The method further includes: The first dimension information and the second dimension information are sent to the second electronic device to instruct the second electronic device to determine the location information based on the first dimension information and the second dimension information.

23. The method of any one of claims 20-22, wherein, The first electronic device is equipped with a rotation module; the method further includes: The scanning plane is formed by the rotation module.

24. The method of any one of claims 20-23, wherein, The first electronic device is configured with a transmitting module and a receiving module, wherein the transmitting module includes multiple transmitting units and / or the receiving module includes multiple receiving units; The emitted beam includes: a beam that forms the scanning plane by being emitted through the plurality of emitted units; The process of acquiring the reflected beam formed by the reflection of the light beam by the target object includes: receiving the reflected beam through the plurality of receiving units.

25. The method of any one of claims 20-24, wherein, The first electronic device is installed independently on the second electronic device, or the first electronic device is integrated into the second electronic device.

26. The method of claim 25, wherein, The first electronic device is located in the middle of the upper frame of the second electronic device.

27. The method according to any one of claims 20-26, characterized in that, The target object can be an active device or a passive object. The active device includes a stylus or a touch glove, and the passive object includes a finger. And / or, The first electronic device is any one of the following: direct time-of-flight (DTOF) radar, indirect time-of-flight (ITOF) radar, binocular radar, structured light radar, and electronically scanned solid-state lidar. And / or, The second electronic device is a smart screen.

28. The method of any one of claims 20-27, wherein, The beam is a laser beam or an infrared beam.

29. An interaction method, characterized by, The method is applied to a second electronic device, and the method includes: The position information of the intended interactive position of the target object on the display screen of the second electronic device is obtained; wherein, the position information is determined by a light beam emitted by the first electronic device and a reflected light beam formed by the reflection of the light beam by the target object, the light beam forming a scanning plane at a position close to the display screen, the scanning plane being parallel or approximately parallel to the plane of the display screen. Based on the location information, execute the interactive operation corresponding to the location information.

30. The method of claim 29, wherein, The location information includes first location information; the step of executing the interactive operation corresponding to the location information includes: Based on the first location information, the user's touch operation is performed on the second electronic device and / or corresponding display elements are presented on the second electronic device.

31. The method of claim 29 or 30, wherein, The location information includes multiple second location information; the step of executing the interactive operation corresponding to the location information includes: Based on the plurality of second location information, a trajectory corresponding to the user's interactive operation is presented on the second electronic device.

32. The method of any one of claims 29-31, wherein, The step of obtaining the location information of the intended interaction position of the target object on the display screen of the second electronic device includes: Receive first-dimensional information and second-dimensional information sent by the first electronic device; wherein, the first-dimensional information is determined based on the emission time of the light beam and the reception time of the reflected light beam, and the second-dimensional information is the angle information of the intentional interaction position relative to the first electronic device; The location information is determined based on the first dimension information and the second dimension information.

33. The method of claim 32, wherein, Determining the location information based on the first dimension information and the second dimension information includes: Based on information associated with the interactive area on the display screen, information is filtered according to the first dimension information and / or the second dimension information to obtain the location information.

34. The method of any one of claims 29-33, wherein, The target object is an active device, including a pressure sensor; the step of executing the interactive operation corresponding to the location information includes: Receive pressure data detected by the pressure sensor sent by the target object; Based on the pressure data, perform the interactive operation corresponding to the location information.

35. The method of claim 34, wherein, The step of executing the interactive operation corresponding to the location information based on the pressure data includes: Based on the pressure data, adjust the presentation effect of the display element of the second electronic device located at the first position information.

36. The method of claim 35, wherein, The presentation effects include the thickness of the presentation trajectory or the animation effects of the display elements.

37. The method of any one of claims 29-36, wherein, The target object is an active device, and the target object includes an attitude sensor. The attitude sensor is used to acquire the attitude data of the target object, and the attitude data is used to predict the handwriting position and / or acquire correction information to determine the position information.

38. The method of claim 37, wherein, The step of obtaining the location information of the intended interaction position of the target object on the display screen of the second electronic device includes: Receive the attitude data; Based on the attitude data, the correction information is obtained to determine the position information.

39. The method of any of claims 29-38, wherein, The target object includes multiple target objects.

40. The method of claim 39, wherein, The location information includes location information of the multiple target objects at multiple intentional interaction locations on the display screen; The step of executing the interactive operation corresponding to the location information includes: Based on the location information of the multiple intentional interaction locations, execute the multiple interactive operations corresponding to the location information of the multiple intentional interaction locations.

41. The method of any one of claims 29-40, wherein, The first electronic device is installed independently on the second electronic device, or the first electronic device is integrated into the second electronic device.

42. The method according to claim 41, characterized in that, The first electronic device is located in the middle of the upper frame of the second electronic device.

43. The method according to any one of claims 29-42, characterized in that, The target object can be an active device or a passive object. The active device includes a stylus or a touch glove, and the passive object includes a finger. And / or, The first electronic device is any one of the following: direct time-of-flight (DTOF) radar, indirect time-of-flight (ITOF) radar, binocular radar, structured light radar, and electronically scanned solid-state lidar. And / or, The second electronic device is a smart screen.

44. The method of any of claims 29-43, wherein, The beam is a laser beam or an infrared beam.

45. An electronic device, comprising: include: A processor and a memory, the memory being coupled to the processor, the memory being used to store computer program code, the computer program code including computer instructions, which, when the processor reads the computer instructions from the memory, cause the electronic device to perform the method as described in any one of claims 20-28, or cause the electronic device to perform the method as described in any one of claims 29-44.

46. The electronic device of claim 45, wherein, The electronic device is any one of direct time-of-flight (DTOF) radar, indirect time-of-flight (ITOF) radar, binocular radar, structured light radar, and electronically scanned solid-state lidar; the electronic device also includes a transmitting module and a receiving module, the transmitting module being used to transmit a light beam, and the receiving module being used to receive a reflected light beam formed by the reflection of the light beam by the target object.

47. A computer-readable storage medium, comprising: The computer-readable storage medium includes a computer program that, when executed on an electronic device, causes the electronic device to perform the method as described in any one of claims 20-28, or causes the electronic device to perform the method as described in any one of claims 29-44.

48. A computer program product, characterised in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 20-28; or, causes the computer to perform the method as described in any one of claims 29-44.

Images