Interaction methods, devices, electronic devices and display devices

Abstract

This disclosure provides an interaction method, apparatus, electronic device, and display device. The interaction method includes: receiving a confirmation message of a control command sent by a second device, the confirmation message including location information of a first location, wherein the second device displays a first virtual screen corresponding to a first interface of a first device, and the first location is the location of the intersection of a virtual identifier and the first virtual screen; determining location information of a second location in the first interface corresponding to the first location based on the location information of the first location and target display parameters, wherein the target display parameters are used to characterize the difference between the display parameters of the first device and the second device; and executing an interaction event triggered for the second location based on the location information of the second location.

Description

Technical Field

[0001] This disclosure relates to the field of augmented reality technology, and more specifically, to an interaction method, apparatus, electronic device, and display device. Background Technology

[0002] With the continuous development of near-eye display technology, the applications of near-eye displays are becoming increasingly widespread. For example, based on wireless networks, display data from terminal devices can be transmitted to near-eye displays to view the terminal device's screen.

[0003] However, due to the limitations of existing near-eye display devices' hardware, the displayed images experience delays, impacting the user experience. Furthermore, near-eye display devices do not support touch functionality, preventing users from directly interacting with the displayed images and thus hindering control of the device, resulting in a poor user experience. Summary of the Invention

[0004] The purpose of this disclosure is to provide an interaction method, apparatus, electronic device, and display device.

[0005] According to a first aspect of this disclosure, an interaction method is provided, the method comprising:

[0006] A confirmation message for receiving a control command sent by a second device is received. The confirmation message includes location information of a first location, wherein the second device displays a first virtual screen corresponding to a first interface of the first device, and the first location is the location of the intersection of the virtual identifier and the first virtual screen.

[0007] Based on the location information of the first location and the target display parameters, the location information of the second location corresponding to the first location in the first interface is determined, wherein the target display parameters are used to characterize the difference between the display parameters of the first device and the second device;

[0008] Based on the location information of the second location, execute the interactive event triggered for the second location.

[0009] Optionally, before determining the location information of the second location corresponding to the first location in the first interface based on the location information of the first location and the target display parameters, the method further includes:

[0010] Obtain the first display parameters of the second device;

[0011] The target display parameters are determined based on the second display parameters of the first device and the first display parameters.

[0012] Optionally, the first display parameters include a first width and a first height of the first virtual image, the second display parameters include a second width and a second height of the screen of the first device, and the target display parameters include a first scaling factor and a second scaling factor; determining the target display parameters based on the second display parameters and the first display parameters of the first device includes:

[0013] The first scaling factor is determined based on the first width and the second width;

[0014] The second proportional coefficient is determined based on the first height and the second height.

[0015] Optionally, after determining the target display parameters based on the second display parameters and the first display parameters of the first device, the method further includes:

[0016] Upon detecting a screen rotation event, the second display parameter is updated to obtain the third display parameter;

[0017] Based on the third display parameter and the first display parameter, the target display parameter is re-determined to obtain the updated target display parameter.

[0018] Optionally, after executing the interactive event triggered at the second location based on the location information of the second location, the method further includes:

[0019] In response to the interaction event, the first interface is updated to obtain the second interface;

[0020] Generate a second virtual screen corresponding to the second interface;

[0021] The second virtual screen is sent to the second device so that the first virtual screen displayed on the second device is updated to the second virtual screen.

[0022] Optionally, before receiving the confirmation message of the control command sent by the second device, the method further includes:

[0023] Obtain the current attitude information of the first device, wherein the current attitude information is used to characterize the first direction of the first device, and the first direction is the orientation of the first device;

[0024] The current posture information is sent to the second device so that the second device can convert the first direction to obtain a second direction, and generate and display a virtual icon according to the second direction, wherein the virtual icon extends along the second direction and points to the first virtual screen.

[0025] According to a second aspect of this disclosure, an interaction method is provided, the method comprising:

[0026] When displaying a first virtual screen corresponding to a first interface of a first device, a confirmation message of a control command is sent to the first device. The confirmation message includes location information of a first position, so that the first device determines the location information of a second position corresponding to the first position in the first interface based on the location information of the first position and the target display parameters, and executes an interactive event triggered for the second position.

[0027] Wherein, the first position is the location of the intersection of the virtual identifier and the first virtual screen, and the target display parameter is used to characterize the difference in display parameters between the first device and the second device.

[0028] Optionally, before sending the confirmation message of the control command to the first device, the method further includes:

[0029] Based on the communication connection with the first device, the first display parameters of the second device are sent to the first device so that the first device can determine the target display parameters according to the first display parameters and the second display parameters of the first device.

[0030] Optionally, after sending the confirmation message of the control command to the first device, the method further includes:

[0031] Receive a second virtual screen sent by the first device, wherein the second virtual screen corresponds to a second interface of the first device after responding to the interaction event;

[0032] Update the first virtual screen to the second virtual screen.

[0033] Optionally, before sending the confirmation message of the control command to the first device, the method further includes:

[0034] Receive current attitude information sent by the first device, wherein the current attitude information is used to characterize the first direction of the first device, and the first direction is the orientation of the first device;

[0035] The first direction is converted to obtain a second direction, and a virtual identifier is generated and displayed according to the second direction, wherein the virtual identifier extends along the second direction and points to the first virtual screen.

[0036] According to a third aspect of this disclosure, an interactive device is provided, comprising:

[0037] The receiving module is used to receive a confirmation message of a control command sent by the second device. The confirmation message includes location information of a first location, wherein the second device displays a first virtual screen corresponding to a first interface of the first device, and the first location is the location of the intersection of the virtual identifier and the first virtual screen.

[0038] The first determining module is used to determine the location information of the second location corresponding to the first location in the first interface based on the location information of the first location and the target display parameters, wherein the target display parameters are used to characterize the difference between the display parameters of the first device and the second device;

[0039] The execution module is used to execute interactive events triggered at the second location based on the location information of the second location.

[0040] According to a fourth aspect of this disclosure, an interactive device is provided, comprising:

[0041] The first sending module is used to send a confirmation message of control command to the first device when displaying a first virtual screen corresponding to the first interface of the first device. The confirmation message includes the location information of the first position, so that the first device can determine the location information of the second position corresponding to the first position in the first interface according to the location information of the first position and the target display parameters, and execute the interactive event triggered for the second position.

[0042] Wherein, the first position is the location of the intersection of the virtual identifier and the first virtual screen, and the target display parameter is used to characterize the difference in display parameters between the first device and the second device.

[0043] According to a fifth aspect of this disclosure, an electronic device is provided, comprising:

[0044] Memory is used to store executable computer instructions;

[0045] A processor configured to execute the interaction method according to the first aspect of this disclosure, under the control of the executable computer instructions.

[0046] According to a sixth aspect of this disclosure, a display device is provided, comprising:

[0047] Memory is used to store executable computer instructions;

[0048] A processor configured to execute the interaction method according to the second aspect of this disclosure, under the control of the executable computer instructions.

[0049] According to a seventh aspect of this disclosure, a computer-readable storage medium is provided that stores computer instructions that can be read and executed by a computer, wherein when the computer instructions are executed by a processor, they perform the interaction method described in the first aspect of this disclosure, or perform the interaction method described in the second aspect of this disclosure.

[0050] According to an embodiment of this application, the location information of a first location sent by a second device according to a control command is received. Based on the location information of the first location and target display parameters, the location information of a second location corresponding to the first location in the first virtual screen displayed by the second device is determined in the first interface displayed by the first device. Based on the location information of the second location, an interactive event triggered at the second location is executed. Thus, this embodiment utilizes the wireless streaming function of the second device to display the screen content of the first device as an extended screen, reducing screen latency and obtaining a higher quality display. Furthermore, it converts user operations on the virtual screen displayed by the second device into actual operations on the display interface of the first device, expanding the functionality of the second device and improving the user experience. Moreover, converting the first location using target display parameters before identifying the interactive event improves the accuracy of interactive event recognition and enables the application of wireless streaming function between devices with different display parameters, making it more convenient to use.

[0051] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as limiting the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram of the hardware configuration of an interactive system that can be used to implement the interactive method of one embodiment;

[0054] Figure 2 This is a flowchart illustrating an interaction method according to one embodiment;

[0055] Figure 3 This is a schematic diagram of a virtual identifier according to one embodiment;

[0056] Figure 4 This is a flowchart illustrating an interaction method according to another embodiment;

[0057] Figure 5This is a schematic block diagram of an interactive device according to one embodiment;

[0058] Figure 6 This is a schematic block diagram of an interactive device according to another embodiment;

[0059] Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to one embodiment;

[0060] Figure 8 This is a schematic diagram of the hardware structure of a display device according to one embodiment. Detailed Implementation

[0061] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0062] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0063] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0064] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0065] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0066] Hereinafter, various embodiments and examples according to the present disclosure will be described with reference to the accompanying drawings.

[0067] <Hardware Configuration>

[0068] Figure 1 This is a schematic diagram of the hardware configuration of an interactive system that can be used to implement an interactive method of one embodiment. Figure 1A first device 100, a second device 200, and a network 300 are illustrated. The first device 100 can connect to the network 300 and can also connect to the second device 200 via a communication method such as Bluetooth. In one embodiment, the first device 100 connects to the second device 200 only via a communication method such as Bluetooth. Multiple servers 301 and 302 can be configured in the network 300. The network 300 can be a wireless communication network or a wired communication network. The network 300 can be a local area network (LAN) or a wide area network (WAN). The network 300 can be for short-range communication or long-range communication.

[0069] In one embodiment, such as Figure 1 As shown, the first device 100 may include a processor 101 and a memory 102. The first device 100 also includes a communication device 103, a display device 104, a user interface 105, a camera device 106, an audio / video interface 107, and a sensor 108, etc. Furthermore, the first device 100 may also include a power management chip 109 and a battery 110, etc.

[0070] The processor 101 can be any type of processor. The memory 102 can store the underlying software, system software, application software, data, etc., required for the operation of the first device 100. The memory 102 can include various forms of memory, such as ROM, RAM, Flash, etc. The communication device 103 can include, for example, a WiFi communication device, a Bluetooth communication device, a 3G, 4G, or 5G communication device. Through the communication device 103, the first device 100 can be deployed in a network. The display device 104 can be a liquid crystal display, an OLED display, etc. In one example, the display device 104 can be a touchscreen. Users can perform input operations through the display device 104. Furthermore, users can also perform fingerprint recognition, etc., through the touchscreen. The user interface 105 can include a USB interface, a Lightning interface, a keyboard, etc. The camera device 106 can be a single camera or multiple cameras. The audio / video interface 107 can include, for example, a speaker interface, a microphone interface, a video transmission interface such as HDMI, etc. The sensor 108 can include, for example, a gyroscope, an accelerometer, a temperature sensor, a humidity sensor, a pressure sensor, etc. For example, the sensor can determine the attitude information of the first device. The power management chip 109 can be used to manage the power input to the first device 100, and can also manage the battery 110 to ensure maximum utilization efficiency. The battery 110 is, for example, a lithium-ion battery.

[0071] The first device 100 may be a mobile phone, a portable computer, a tablet computer, a handheld computer, a wearable device, etc., and this disclosure does not limit it. Figure 1 The components shown are merely illustrative. The first device 100 may include... Figure 1 One or more of the components shown, but not necessarily including Figure 1 All components in it. Figure 1 The first device 100 shown is merely illustrative and is by no means intended to limit the embodiments, applications, or uses herein.

[0072] In this embodiment, the memory 102 of the first device 100 is used to store program instructions that control the processor 101 to perform interactive methods. Those skilled in the art can design these instructions based on the disclosed scheme of this invention. How the instructions control the processor to perform operations is well known in the art and will not be described in detail here.

[0073] In one embodiment, such as Figure 1 As shown, the second device 200 may include a processor 201 and a memory 202. The second device 200 also includes a communication device 203, a display device 204, a user interface 205, a camera device 206, an audio / video interface 207, and a sensor 208, etc. Furthermore, the second device 200 may also include a power management chip 209 and a battery 210, etc.

[0074] The second device 200 can be a near-eye display device. For example, VR (Virtual Reality) glasses, AR (Augmented Reality) glasses, and MR (Mixed Reality) glasses, etc., which are not limited in this disclosure. Figure 1 The components shown are merely illustrative. The second device 200 may include... Figure 1 One or more of the components shown, but not necessarily including Figure 1 All components in it. Figure 1 The second device 200 shown is merely illustrative and is by no means intended to limit the embodiments, applications, or uses herein.

[0075] In this embodiment, the memory 202 of the second device 200 is used to store program instructions that control the processor 201 to perform interactive methods. Those skilled in the art can design these instructions based on the disclosed scheme of this invention. How the instructions control the processor to perform operations is well known in the art and will not be described in detail here.

[0076] It should be understood that, despite Figure 1 Only one first device 100 and one second device 200 are shown, but this does not mean that the number of each is limited. The interactive system may contain multiple first devices 100 and multiple second devices 200.

[0077] In the above description, those skilled in the art can design instructions based on the solutions provided in this disclosure. How the instructions control the processor to operate is well known in the art, and therefore will not be described in detail here.

[0078] <Method Implementation>

[0079] This disclosure provides an interaction method, which can be provided by... Figure 1 The first device implementation in the illustrated interactive system, such as Figure 2 As shown, the interaction method includes the following steps: steps S2100 to S2300.

[0080] Step S2100: Receive a confirmation message of the control command sent by the second device. The confirmation message includes location information of the first position, wherein the second device displays a first virtual screen corresponding to the first interface of the first device, and the first position is the location of the intersection of the virtual identifier and the first virtual screen.

[0081] In this embodiment, the second device has a wireless streaming function. When the wireless streaming function is enabled on the second device, the second device can act as an extended display device for the first device, used to display the content displayed by the first device. That is, based on the wireless communication connection established between the second device and the first device, the second device can acquire the display content of the first device, generate and display a first virtual screen corresponding to the first interface displayed by the first device. Furthermore, the user can operate based on the first virtual screen displayed by the second device to control the first device.

[0082] In an optional embodiment, the first device may be, for example, a mobile phone, a laptop, a tablet computer, a PDA, a wearable device, etc. The first interface may be the operation interface of an application running on the first device, or it may be the content interface displayed by the first device.

[0083] In an optional embodiment, the second device may be a display device. Exemplarily, the first device may be a near-eye display device. For example, virtual reality glasses, augmented reality glasses, and mixed reality glasses, etc., are not limited to this embodiment. The first virtual image may be a virtual image displayed by the second device corresponding to the first interface of the first device. The first virtual image may be a 3DOF image.

[0084] In this embodiment, the control command can be a command generated by the user interacting with the first virtual screen, used to trigger an interactive event at the intersection location. The control command is used to trigger the second device to obtain the first location information, that is, the location information of the intersection point between the virtual identifier and the first virtual screen.

[0085] Furthermore, the control commands can be triggered based on user input. For example, user input can be gesture input. More specifically, if the second device is equipped with a camera, the second device can recognize the user's gesture input and generate control commands based on the gesture input.

[0086] For example, when the second device and the accessory are in communicative connection, control commands are generated and sent to the second device in response to user input to the accessory. The accessory may be, for example, a gamepad, a mouse, etc.

[0087] For example, when the second device and the third device are in a communicative connection, in response to user input to the third device, the third device generates and sends control commands to the second device. The third device may be a mobile terminal or the like, and may have a target application installed for controlling the second device. User input to the third device may be input to a target area of ​​the third device while the target application is running.

[0088] In this embodiment, the virtual identifier can represent the current posture of the first device. The user can control the first virtual screen based on the virtual identifier. For example, selecting a target object in the first virtual screen. Exemplarily, the virtual identifier can be a virtual ray. This virtual ray can be a straight line. Figure 3 As shown, the virtual ray 31 can also be a curve.

[0089] like Figure 3 As shown, the first position 32 can be the location of the intersection of the virtual identifier and the first virtual screen. That is, the position triggered when the user triggers a control command.

[0090] The following specific example illustrates the process of generating virtual identifiers.

[0091] In one embodiment, before receiving a confirmation message of a control command sent by the second device, the method further includes: obtaining current posture information of the first device, wherein the current posture information is used to characterize a first direction of the first device, the first direction being the orientation of the first device; sending the current posture information to the second device so that the second device converts the first direction to obtain a second direction, and generating and displaying a virtual identifier based on the second direction, wherein the virtual identifier extends along the second direction and points to the first virtual screen.

[0092] In this embodiment, when the first device acts as the device operating the second device, the first device can send the current attitude information to the second device so that the second device can generate and display a virtual identifier.

[0093] In this embodiment, the current posture information of the first device can represent the first orientation of the first device, that is, the orientation of the first device. Taking the first device as a terminal device as an example, it is usually used in a one-handed holding manner, that is, the top of the first device faces outward from the user. Based on this, the first orientation of the first device can specifically be the orientation of the top of the terminal device.

[0094] The coordinate systems of the first and second devices can be the same or different. On the one hand, if the coordinate systems of the first and second devices are different, after determining the first device's first direction based on its current posture information, the first direction needs to be transformed so that the second device can generate and display a virtual icon according to the transformed direction, facilitating user operation of the first virtual screen using the virtual icon. On the other hand, if the coordinate systems of the first and second devices are the same, their operating directions may differ. In this case, after determining the first device's first direction based on its current posture information, the first direction needs to be transformed so that the second device can generate and display a virtual icon according to the transformed direction, facilitating user operation of the first virtual screen using the virtual icon.

[0095] For example, the first direction can be transformed based on a preset transformation relationship to obtain the second direction. When the coordinate systems of the first and second devices are different, the preset transformation relationship can be a correspondence between the coordinate systems of the first and second devices. For example, the coordinate system of the first device is a right-handed coordinate system (Android coordinate system), and the coordinate system of the second device is a left-handed coordinate system. When the coordinate systems of the first and second devices are the same, but their usage directions are different, the preset transformation relationship can be a correspondence between the coordinate systems of the first device in use and the second device in use.

[0096] According to this embodiment, based on the communication connection between the first device and the second device, the current orientation of the first device can be determined and converted so that the second device generates and displays a virtual icon according to the converted orientation. In this way, this embodiment allows the first device to control the first virtual screen displayed on the second device, and by representing the rotation direction of the first device as the extension direction of the virtual icon, the ease of operation of the second device is further improved, thereby enhancing the user experience of the second device.

[0097] After step S2100, step S2200 is executed, in which the position information of the second position corresponding to the first position in the first interface is determined according to the position information of the first position and the target display parameters. The target display parameters are used to characterize the difference between the display parameters of the first device and the second device.

[0098] In this embodiment, when the second device is used as an extended screen to display the content of the first device, the user can only see the first virtual screen of the second device and can only operate within the first virtual screen, rather than directly on the first device. Furthermore, on the one hand, because the display parameters of the first and second devices are different, the size of the first interface displayed on the first device and the first virtual screen displayed on the second device are different, resulting in different pixel positions of the target object in the first virtual screen and in the first interface; on the other hand, different first devices (mobile terminals) can have different screen sizes and display resolutions. Therefore, after obtaining the position information of the first position in the first virtual screen, the position information of the second position corresponding to the first position in the first interface is determined based on the position information of the first position and the target display parameters, so that the interactive event triggered for the second position can be executed according to the position information of the second position.

[0099] In this embodiment, the target display parameter can characterize the difference in display parameters between the first device and the second device. For example, the target display parameter can characterize the difference in display resolution between the first device and the second device.

[0100] In one embodiment, before determining the location information of the second location corresponding to the first location in the first interface based on the location information of the first location and the target display parameters, the method further includes: obtaining the first display parameters of the second device; and determining the target display parameters based on the second display parameters of the first device and the first display parameters.

[0101] In this embodiment, taking a near-eye display device as an example, the first display parameter can be the screen resolution of the second device. The first display parameter can also be the display parameters of the virtual image displayed by the second device. More specifically, the first display parameter can include the first width and first height of the first virtual image.

[0102] Taking the first device as an example of a terminal device, the second display parameter can be the screen resolution of the first device. More specifically, the second display parameter can include the second width and the second height of the screen of the first device.

[0103] In one embodiment, the first display parameter includes a first width and a first height of the first virtual image, the second display parameter includes a second width and a second height of the screen of the first device, and the target display parameter includes a first scaling factor and a second scaling factor. Determining the target display parameter based on the second display parameter and the first display parameter of the first device includes: determining the first scaling factor based on the first width and the second width; and determining the second scaling factor based on the first height and the second height.

[0104] For example, the first scaling factor can be determined based on the first width and the second width, which can be the ratio of the first width and the second width. For instance, the first scaling factor can be determined according to the following formula (1).

[0105] Dx=WidthG / WidthM (1)

[0106] Where Dx is the first scaling factor, WidthG is the first width (the display width of the first virtual screen of the second device), and WidthM is the second width (the screen width of the first device).

[0107] For example, the second proportionality coefficient can be determined based on the first height and the second height, which can be the ratio of the first height to the second height. For instance, the second proportionality coefficient can be determined according to the following formula (2).

[0108] Dy=HeightG / HeightM (2)

[0109] Where Dy is the second scaling factor, HeightG is the first height (the display height of the first virtual screen of the second device), and HeightM is the second height (the screen height of the first device).

[0110] Taking the aforementioned first scaling factor Dx and second scaling factor Dy as examples, the process of determining the position information of the second position is explained. Assuming that when receiving control commands, the coordinates (first position information) of the intersection point of the virtual identifier and the first virtual screen are (Gx, Gy), then the position information of the second position corresponding to the first position in the first interface is (Mx, My), where the horizontal coordinate of the second position Mx = Gx / Dx, and the vertical coordinate of the second position My = Gy / Dy. It should be noted that in this embodiment, the first virtual screen corresponding to the first interface is displayed in full screen.

[0111] According to this embodiment, when using the wireless streaming function of the second device, the first display parameters of the second device can be obtained, and the target display parameters can be determined based on the first display parameters of the second device and the second display parameters of the first device. In this way, when executing an interactive event, the position selected by the user can be accurately determined based on the target display parameters, thereby realizing the interaction between the first device and the second device.

[0112] In one embodiment, after determining the target display parameters based on the second display parameters and the first display parameters of the first device, the method may further include: updating the second display parameters to obtain a third display parameter when a screen rotation event is detected; and re-determining the target display parameters based on the third display parameters and the first display parameters to obtain the updated target display parameters.

[0113] In this embodiment, the screen rotation event can be a user switching the screen orientation of the first device. For example, switching from landscape to portrait mode. Or, for example, switching from portrait to landscape mode.

[0114] In practical implementation, data collected by the accelerometer and gyroscope of the first device can be acquired, and the screen rotation event of the first device can be detected based on the data collected by the accelerometer and gyroscope. This application embodiment does not specifically limit the steps for detecting the screen rotation event.

[0115] The following example illustrates the process of redetermining the target display parameters.

[0116] When the screen of the first device is used in portrait mode, the second display parameters of the first device include the display width (WidthM) and the display height (HeightM). The first scaling factor is Dx = WidthG / WidthM, and the second scaling factor is Dy = HeightG / HeightM.

[0117] When the screen usage mode of the first device is switched to landscape display mode, the display width and display height of the first device are swapped, and the third display parameters of the first device that are re-acquired include the display width (HeightM) and the display height (WidthM). Therefore, the first scaling factor is redefined as Dx = WidthG / HeightM, and the second scaling factor is Dy = HeightG / WidthM.

[0118] According to this embodiment, during the wireless streaming process using the second device and the first device, it is possible to detect in real time whether the screen rotation event of the first device occurs. When the screen rotation event of the first device occurs, the display parameters of the first device are re-determined, and then the target display parameters are re-determined. In this way, the failure of interactive event recognition can be avoided due to the change of the screen display direction of the first device. When displaying the screen content of the first device using the wireless streaming function of the second device, the screen display direction of the first device can be changed for different applications, resulting in a better user experience.

[0119] After step S2200, step S2300 is executed, whereby an interactive event triggered at the second location is executed based on the location information of the second location.

[0120] In this embodiment, the interaction event can be a click event, a long press event, a drag event, etc.

[0121] In one embodiment, after executing an interactive event triggered by the second location based on the location information of the second location, the method may further include: updating the first interface in response to the interactive event to obtain the second interface; generating a second virtual screen corresponding to the second interface; and sending the second virtual screen to the second device so that the first virtual screen displayed on the second device is updated to the second virtual screen.

[0122] According to embodiments of this disclosure, the system receives location information of a first location sent by a second device according to a control command. Based on the location information of the first location and target display parameters, it determines the location information of a second location in the first interface displayed by the first device that corresponds to the first location in the first virtual screen displayed by the second device. Then, based on the location information of the second location, it executes an interactive event triggered at the second location. Thus, this embodiment utilizes the wireless streaming function of the second device to display the content of the first device's screen as an extended screen, resulting in a higher quality display. Furthermore, it converts user operations on the virtual screen displayed by the second device into actual operations on the first device's display interface, expanding the functionality of the second device and improving the user experience. Moreover, converting the first location using target display parameters before identifying interactive events improves the accuracy of interactive event recognition and enables the application of wireless streaming function between devices with different display parameters, making it more convenient to use.

[0123] This disclosure provides an interaction method, which can be provided by... Figure 1 The second device implementation in the illustrated interactive system, such as Figure 4 As shown, the interaction method includes the following steps: Step S4100.

[0124] In step S4100, when a first virtual screen corresponding to the first interface of the first device is displayed, a confirmation message of control command is sent to the first device. The confirmation message includes the location information of the first position, so that the first device can determine the location information of the second position corresponding to the first position in the first interface according to the location information of the first position and the target display parameters, and execute the interactive event triggered for the second position.

[0125] In this embodiment, the second device has a wireless streaming function. When the wireless streaming function is enabled on the second device, the second device can act as an extended display device for the first device, used to display the content displayed by the first device. That is, based on the wireless communication connection established between the second device and the first device, the second device can acquire the display content of the first device, generate and display a first virtual screen corresponding to the first interface displayed by the first device. Furthermore, the user can operate based on the first virtual screen displayed by the second device to control the first device.

[0126] The control command can be a command generated by the user interacting with the first virtual screen, used to trigger an interactive event at the intersection location. The control command is used to trigger the second device to obtain the first location information, that is, the location information of the intersection point between the virtual identifier and the first virtual screen.

[0127] The first position can be the location where the virtual icon intersects with the first virtual screen. In other words, it is the position triggered when the user triggers a control command.

[0128] The target display parameter is used to characterize the difference in display parameters between the first device and the second device. For example, the target display parameter can characterize the difference in display resolution between the first device and the second device.

[0129] In one embodiment, before sending a confirmation message for the control command to the first device, the method may further include: receiving current posture information sent by the first device, wherein the current posture information is used to characterize a first direction of the first device, the first direction being the orientation of the first device; converting the first direction to obtain a second direction, and generating and displaying a virtual identifier based on the second direction, wherein the virtual identifier extends along the second direction and points to a first virtual screen.

[0130] In this embodiment, when the first device acts as the device operating the second device, the first device can send the current attitude information to the second device so that the second device can generate and display a virtual identifier.

[0131] In this embodiment, the current posture information of the first device can represent the first orientation of the first device, that is, the orientation of the first device. Taking the first device as a terminal device as an example, it is usually used in a one-handed holding manner, that is, the top of the first device faces outward from the user. Based on this, the first orientation of the first device can specifically be the orientation of the top of the terminal device.

[0132] According to this embodiment, based on the communication connection between the first device and the second device, the current orientation of the first device can be determined and converted so that the second device generates and displays a virtual icon according to the converted orientation. In this way, this embodiment allows the first device to control the first virtual screen displayed on the second device, and by representing the rotation direction of the first device as the extension direction of the virtual icon, the ease of operation of the second device is further improved, thereby enhancing the user experience of the second device.

[0133] In one embodiment, before sending a confirmation message for the control command to the first device, the method may further include: sending a first display parameter of the second device to the first device based on the communication connection with the first device, so that the first device can determine the target display parameter according to the first display parameter and the second display parameter of the first device.

[0134] In this embodiment, taking a near-eye display device as an example, the first display parameter can be the screen resolution of the second device. The first display parameter can also be the display parameters of the virtual image displayed by the second device. More specifically, the first display parameter can include the first width and first height of the first virtual image.

[0135] Taking the first device as an example of a terminal device, the second display parameter can be the screen resolution of the first device. More specifically, the second display parameter can include the second width and the second height of the screen of the first device.

[0136] In one embodiment, after sending a confirmation message for the control command to the first device, the method may further include: receiving a second virtual screen sent by the first device, wherein the second virtual screen corresponds to a second interface of the first device after responding to an interaction event; and updating the first virtual screen to the second virtual screen.

[0137] According to embodiments of this disclosure, when displaying a first virtual screen corresponding to a first interface of a first device, a confirmation message for a control command is sent to the first device. The confirmation message includes location information of a first position, enabling the first device to determine the location information of a second position corresponding to the first position on the first interface based on the location information of the first position and target display parameters, and to execute an interactive event triggered by the second position. Thus, this embodiment utilizes the wireless streaming function of the second device to display the screen content of the first device as an extended screen, reducing screen latency and obtaining a higher quality display. Furthermore, it converts user operations on the virtual screen displayed on the second device into actual operations on the display interface of the first device, expanding the functionality of the second device and improving the user experience. Moreover, converting the first position using target display parameters before recognizing interactive events improves the accuracy of interactive event recognition and allows for the application of wireless streaming functions between devices with different display parameters, making it more convenient to use.

[0138] The following is a specific example to illustrate the interaction method.

[0139] Step S501: Establish a communication connection between the first device (mobile terminal) and the second device (display device);

[0140] In step S502, the first device (mobile terminal) sends a first virtual screen corresponding to the first interface to the second device (display device);

[0141] In step S503, the second device (display device) displays the first virtual screen;

[0142] Step S504: Based on the communication connection established between the third device (the terminal device for controlling the second device) and the second device (the display device), the third device sends the current attitude information of the third device to the second device, wherein the current attitude information is used to characterize the first orientation of the third device;

[0143] In step S505, the second device (display device) converts the first direction to obtain the second direction, and generates and displays a virtual logo based on the second direction, wherein the virtual logo extends along the second direction and points to the first virtual screen;

[0144] In step S506, the third device responds to the user's input and sends a control command to the second device;

[0145] Step S507: The second device sends a confirmation message of the control command to the first device. The confirmation message includes the location information of the first location, which is the location of the intersection of the virtual identifier and the first virtual screen.

[0146] Step S508: The first device determines the position information of the second position corresponding to the first position in the first interface based on the position information of the first position and the target display parameters;

[0147] Step S509: The first device executes the interactive event triggered for the second location based on the location information of the second location;

[0148] In step S510, the first device responds to the interaction event, updates the first interface to obtain the second interface, and sends the second virtual screen corresponding to the second interface to the second device;

[0149] Step S511: The second device updates the first virtual screen to the second virtual screen.

[0150] According to embodiments of this disclosure, by utilizing the wireless streaming function of the second device to display the screen content of the first device as an extended screen, screen latency can be reduced, resulting in a higher quality display. Furthermore, user operations on the virtual screen displayed on the second device can be converted into actual operations on the display interface of the first device, expanding the functionality of the second device and improving the user experience. Moreover, by converting the first position using target display parameters before recognizing interactive events, the accuracy of interactive event recognition can be improved. This allows for the application of wireless streaming function between devices with different display parameters, making it more convenient to use.

[0151] <Device Embodiment>

[0152] This disclosure provides an interactive device applied to a first device, such as... Figure 5 As shown, the interactive device 600 may include a receiving module 601, a first determining module 602, and an execution module 603.

[0153] The receiving module 601 is used to receive a confirmation message of a control command sent by the second device. The confirmation message includes location information of a first location, wherein the second device displays a first virtual screen corresponding to a first interface of the first device, and the first location is the location of the intersection of the virtual identifier and the first virtual screen.

[0154] The first determining module 602 is used to determine the location information of the second position corresponding to the first position in the first interface based on the location information of the first position and the target display parameters, wherein the target display parameters are used to characterize the difference between the display parameters of the first device and the second device.

[0155] The execution module 603 is used to execute the interactive event triggered for the second location based on the location information of the second location.

[0156] In one embodiment, the interactive device 600 may further include:

[0157] The first acquisition module is used to acquire the first display parameters of the second device.

[0158] The second determining module is used to determine the target display parameters based on the second display parameters of the first device and the first display parameters.

[0159] In one embodiment, the first display parameters include a first width and a first height of the first virtual image, the second display parameters include a second width and a second height of the screen of the first device, and the target display parameters include a first scaling factor and a second scaling factor; the second determining module may further include:

[0160] The first determining unit is configured to determine the first proportional coefficient based on the first width and the second width.

[0161] The second determining unit is used to determine the second proportional coefficient based on the first height and the second height.

[0162] In one embodiment, the interactive device 600 may further include:

[0163] The display parameter update module is used to update the second display parameter to obtain the third display parameter when a screen rotation event is detected.

[0164] The third determining module is used to redetermine the target display parameters based on the third display parameters and the first display parameters to obtain the updated target display parameters.

[0165] In one embodiment, the interactive device 600 may further include:

[0166] The interface update module is used to update the first interface in response to the interaction event, so as to obtain the second interface.

[0167] The generation module is used to generate a second virtual screen corresponding to the second interface.

[0168] The first sending module is used to send the second virtual screen to the second device so that the first virtual screen displayed on the second device is updated to the second virtual screen.

[0169] In one embodiment, the interactive device 600 may further include:

[0170] The second acquisition module is used to acquire the current posture information of the first device, wherein the current posture information is used to characterize the first direction of the first device, and the first direction is the orientation of the first device.

[0171] The second sending module is used to send the current posture information to the second device, so that the second device can convert the first direction to obtain a second direction, and generate and display a virtual identifier according to the second direction, wherein the virtual identifier extends along the second direction and points to the first virtual screen.

[0172] According to embodiments of this disclosure, the system receives location information of a first location sent by a second device according to a control command. Based on the location information of the first location and target display parameters, it determines the location information of a second location in the first interface displayed by the first device that corresponds to the first location in the first virtual screen displayed by the second device. Then, based on the location information of the second location, it executes an interactive event triggered at the second location. Thus, this embodiment utilizes the wireless streaming function of the second device to display the content of the first device's screen as an extended screen, resulting in a higher quality display. Furthermore, it converts user operations on the virtual screen displayed by the second device into actual operations on the first device's display interface, expanding the functionality of the second device and improving the user experience. Moreover, converting the first location using target display parameters before identifying interactive events improves the accuracy of interactive event recognition and enables the application of wireless streaming function between devices with different display parameters, making it more convenient to use.

[0173] This disclosure also provides another interactive device, applied to a second device, such as... Figure 6 As shown, the interactive device 700 may include a first sending module 701.

[0174] The first sending module 701 is used to send a confirmation message of a control command to the first device when displaying a first virtual screen corresponding to the first interface of the first device. The confirmation message includes the location information of the first position, so that the first device can determine the location information of the second position corresponding to the first position in the first interface according to the location information of the first position and the target display parameters, and execute the interactive event triggered for the second position.

[0175] Wherein, the first position is the location of the intersection of the virtual identifier and the first virtual screen, and the target display parameter is used to characterize the difference in display parameters between the first device and the second device.

[0176] In one embodiment, the interactive device 700 may further include:

[0177] The second sending module is configured to send the first display parameters of the second device to the first device based on the communication connection with the first device, so that the first device can determine the target display parameters according to the first display parameters and the second display parameters of the first device.

[0178] In one embodiment, the interactive device 700 may further include:

[0179] The first receiving module is configured to receive a second virtual screen sent by the first device, wherein the second virtual screen corresponds to a second interface of the first device after responding to the interaction event.

[0180] An update module is used to update the first virtual screen to the second virtual screen.

[0181] In one embodiment, the interactive device 700 may further include:

[0182] The second receiving module is used to receive current attitude information sent by the first device, wherein the current attitude information is used to characterize a first direction of the first device, and the first direction is the orientation of the first device.

[0183] The generation module is used to convert the first direction to obtain a second direction, and generate and display a virtual identifier based on the second direction, wherein the virtual identifier extends along the second direction and points to the first virtual screen.

[0184] According to embodiments of this disclosure, when displaying a first virtual screen corresponding to a first interface of a first device, a confirmation message for a control command is sent to the first device. The confirmation message includes location information of a first position, enabling the first device to determine the location information of a second position corresponding to the first position on the first interface based on the location information of the first position and target display parameters, and to execute an interactive event triggered by the second position. Thus, this embodiment utilizes the wireless streaming function of the second device to display the screen content of the first device as an extended screen, reducing screen latency and obtaining a higher quality display. Furthermore, it converts user operations on the virtual screen displayed on the second device into actual operations on the display interface of the first device, expanding the functionality of the second device and improving the user experience. Moreover, converting the first position using target display parameters before recognizing interactive events improves the accuracy of interactive event recognition and allows for the application of wireless streaming functions between devices with different display parameters, making it more convenient to use.

[0185] <Equipment Example>

[0186] Figure 7 This is a schematic diagram of the hardware structure of an electronic device according to one embodiment. For example... Figure 7 As shown, the electronic device 800 includes a memory 801 and a processor 802.

[0187] The memory 801 can be used to store executable computer instructions.

[0188] The processor 802 can be used to execute the interactive method according to the method embodiments of this disclosure, under the control of the executable computer instructions.

[0189] In another embodiment, the electronic device 800 may include the above-mentioned interactive device 600.

[0190] In one embodiment, each module of the above-mentioned interactive device 600 can be implemented by the processor 802 running computer instructions stored in the memory 801.

[0191] In one embodiment, the electronic device 800 may be a mobile phone, a portable computer, a tablet computer, a PDA, a wearable device, etc.

[0192] Figure 8 This is a schematic diagram of the hardware structure of a display device according to one embodiment. For example... Figure 8 As shown, the display device 900 includes a memory 901 and a processor 902.

[0193] The memory 901 can be used to store executable computer instructions.

[0194] The processor 902 can be used to execute the interactive method according to the method embodiments of this disclosure, under the control of the executable computer instructions.

[0195] In another embodiment, the display device 900 may include the above-mentioned interactive device 700.

[0196] In one embodiment, each module of the above-mentioned interactive device 700 can be implemented by the processor 902 running computer instructions stored in the memory 901.

[0197] In one embodiment, the display device 900 may be a near-eye display device. This near-eye display device may be, for example, VR glasses, AR glasses, or MR glasses.

[0198] According to embodiments of this disclosure, by utilizing the wireless streaming function of the second device to display the screen content of the first device as an extended screen, screen latency can be reduced, resulting in a higher quality display. Furthermore, user operations on the virtual screen displayed on the second device can be converted into actual operations on the display interface of the first device, expanding the functionality of the second device and improving the user experience. Moreover, by converting the first position using target display parameters before recognizing interactive events, the accuracy of interactive event recognition can be improved. This allows for the application of wireless streaming function between devices with different display parameters, making it more convenient to use.

[0199] Computer-readable storage media

[0200] This disclosure also provides a computer-readable storage medium storing computer instructions thereon, which, when executed by a processor, perform the interactive method provided in this disclosure.

[0201] Embodiments of this disclosure may be systems, methods, and / or computer program products. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the embodiments of this disclosure.

[0202] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0203] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0204] Computer program instructions used to perform the operations of embodiments of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of embodiments of this disclosure.

[0205] Various aspects of embodiments of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0206] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0207] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0208] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0209] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the embodiments of this disclosure is defined by the appended claims.

Claims

1. An interaction method, characterized in that, The method includes: Obtain the current posture information of the first device, wherein the current posture information is used to characterize the first direction of the first device, and the first direction is the orientation of the first device; The current posture information is sent to the second device, so that the second device converts the first direction to obtain a second direction, and generates and displays a virtual icon according to the second direction, wherein the virtual icon extends along the second direction and points to the first virtual screen; A confirmation message for receiving a control command sent by the second device is received. The confirmation message includes location information of a first location, wherein the second device displays a first virtual screen corresponding to a first interface of the first device, and the first location is the location of the intersection of the virtual identifier and the first virtual screen. Based on the location information of the first location and the target display parameters, the location information of the second location corresponding to the first location in the first interface is determined, wherein the target display parameters are used to characterize the difference between the display parameters of the first device and the second device; Based on the location information of the second location, execute the interactive event triggered for the second location.

2. The method according to claim 1, characterized in that, Before determining the location information of the second location corresponding to the first location in the first interface based on the location information of the first location and the target display parameters, the method further includes: Obtain the first display parameters of the second device; The target display parameters are determined based on the second display parameters of the first device and the first display parameters.

3. The method according to claim 2, characterized in that, The first display parameters include a first width and a first height of the first virtual image; the second display parameters include a second width and a second height of the screen of the first device; and the target display parameters include a first scaling factor and a second scaling factor. Determining the target display parameters based on the second display parameters and the first display parameters of the first device includes: The first scaling factor is determined based on the first width and the second width; The second proportional coefficient is determined based on the first height and the second height.

4. The method according to claim 2, characterized in that, After determining the target display parameters based on the second display parameters and the first display parameters of the first device, the method further includes: Upon detecting a screen rotation event, the second display parameter is updated to obtain the third display parameter; Based on the third display parameter and the first display parameter, the target display parameter is re-determined to obtain the updated target display parameter.

5. The method according to claim 1, characterized in that, After executing the interactive event triggered at the second location based on the location information of the second location, the method further includes: In response to the interaction event, the first interface is updated to obtain the second interface; Generate a second virtual screen corresponding to the second interface; The second virtual screen is sent to the second device so that the first virtual screen displayed on the second device is updated to the second virtual screen.

6. An interaction method, characterized in that, The method includes: When displaying a first virtual screen corresponding to a first interface of the first device, the system receives current posture information sent by the first device, wherein the current posture information is used to characterize a first direction of the first device, and the first direction is the orientation of the first device; The first direction is converted to obtain a second direction, and a virtual identifier is generated and displayed according to the second direction, wherein the virtual identifier extends along the second direction and points to the first virtual screen; A confirmation message for sending a control command to a first device is sent. The confirmation message includes the location information of a first location, so that the first device can determine the location information of a second location in the first interface corresponding to the first location based on the location information of the first location and the target display parameters, and execute the interactive event triggered for the second location. Wherein, the first position is the location of the intersection of the virtual identifier and the first virtual image, and the target display parameter is used to characterize the difference in display parameters between the first device and the second device.

7. The method according to claim 6, characterized in that, Before sending the confirmation message of the control command to the first device, the method further includes: Based on the communication connection with the first device, the first display parameters of the second device are sent to the first device so that the first device can determine the target display parameters according to the first display parameters and the second display parameters of the first device.

8. The method according to claim 6, characterized in that, After sending the confirmation message of the control command to the first device, the method further includes: Receive a second virtual screen sent by the first device, wherein the second virtual screen corresponds to a second interface of the first device after responding to the interaction event; Update the first virtual screen to the second virtual screen.

9. An interactive device, characterized in that, include: The receiving module is used to receive a confirmation message of a control command sent by the second device. The confirmation message includes location information of a first location, wherein the second device displays a first virtual screen corresponding to a first interface of the first device, and the first location is the location of the intersection of the virtual identifier and the first virtual screen. The first determining module is used to determine the location information of the second location corresponding to the first location in the first interface based on the location information of the first location and the target display parameters, wherein the target display parameters are used to characterize the difference between the display parameters of the first device and the second device; The execution module is used to execute interactive events triggered at the second location based on the location information of the second location; The second acquisition module is used to acquire the current posture information of the first device, wherein the current posture information is used to represent the first direction of the first device, and the first direction is the orientation of the first device; The second sending module is used to send the current posture information to the second device, so that the second device can convert the first direction to obtain a second direction, and generate and display a virtual identifier according to the second direction, wherein the virtual identifier extends along the second direction and points to the first virtual screen.

10. An interactive device applied to a near-eye display device, characterized in that, include: The first sending module is used to send a confirmation message of control command to the first device when displaying a first virtual screen corresponding to the first interface of the first device. The confirmation message includes the location information of the first position, so that the first device can determine the location information of the second position corresponding to the first position in the first interface according to the location information of the first position and the target display parameters, and execute the interactive event triggered for the second position. Wherein, the first position is the location of the intersection of the virtual identifier and the first virtual image, and the target display parameter is used to characterize the difference in display parameters between the first device and the second device; The second receiving module is used to receive current attitude information sent by the first device, wherein the current attitude information is used to characterize the first direction of the first device, and the first direction is the orientation of the first device; The generation module is used to convert the first direction to obtain a second direction, and generate and display a virtual identifier based on the second direction, wherein the virtual identifier extends along the second direction and points to the first virtual screen.

11. An electronic device, characterized in that, include: Memory is used to store executable computer instructions; A processor configured to execute the interactive method according to any one of claims 1-5, under the control of the executable computer instructions.

12. A display device, characterized in that, include: Memory is used to store executable computer instructions; A processor configured to execute the interactive method according to any one of claims 6-8, under the control of the executable computer instructions.

Images